SDMX-HD (Health Domain) v1.0 documentation
                                        Version 2009.04.WHO.03

Getting Started

Setup the environment

  1. Create a folder called "SDMX" as your working folder in which all SDMX-HD files will be placed.
  2. Unzip the "SDMX-HD.v1.0.zip" file into the "SDMX" folder. You should now see the following folder structure:

    • SDMX-HD.v1.0 MyWork - This folder is your working area in which you will create the SDMX-HD files to exchange with other systems.
    • SDMX-HD.v1.0 sample1 - This folder contains an example of SDMX-HD files used to exchange indicator definitions.
    • SDMX-HD.v1.0 sample2 - This folder contains an example of SDMX-HD files used to exchange indicator definitions and data.
    • SDMX-HD.v1.0 Testbed - This folder is created by the "CreateTESTBED.bat" file which copies SDMX-HD.v1.0 sample1 or 2 + SDMX-HD-v1.0 folders into a unique folder to resolve cross-references between files (*.xsd and *.xml SDMX files referrenced as external)
    • SDMX-HD-v1.0 - This folder contains the SDMX-HD v1.0 common and custom artifacts used as building blocks for SDMX-HD messages. .. ..

Exchanging indicator definitions

Setting up the MyWork folder
  1. Copy the sub folders and files of the "SDMX-HD-v1.0" folder to the "SDMX-HD.v1.0 MyWork" folder so you have a folder structure like this:
    The following files are not needed in your "SDMX-HD.v1.0 MyWork" folder as they are not required in the SDMX-HD message (delete them if you wish):
    • GettingStarted.html
    • index.html
    • foldercontents.xml
    • foldercontents.xsl

  2. Rename "TemplateDSD.xml" file to DSD.xml
  3. Rename "TemplateMSD.xml" file to MSD.xml
  4. Rename "TemplateGMS.xml" file to GMS.xml
  5. Rename "TemplateCDS.xml" file to CDS.xml (only needed if data is transmitted)

Create your DSD.xml file Codelists
  1. Open the DSD.xml file with the XML editor of your choice.
  2. Open the "COMMON/SDMX-HD/v1.0/organisationschemes/ORGS_COMMON+SDMX-HD+1.0.xml" file and check your organization is defined in the list. Add it if it is not!
  3. Open the "additional/CL_INDICATOR.xml" codelist and populate it with the name of the indicators to exchange
  4. Update any common codelist that have isFinal=false with any items you need (i.e. that are referrenced from another file)
  5. Create any "custom" codelist you need in the "CUSTOM/WHO/v1.0/codelists" folder.
  6. Add these new custom codelist to the DSD file e.g here is custom Age Group codelist:

    <structure:CodeList id="CL_AGROUP" agencyID="WHO" version="1.0" isFinal="false" urn="urn:sdmx:org.sdmx.infomodel.codelist=WHO:CL_AGROUP" isExternalReference="true" uri="./CUSTOM/WHO/v1.0/codelists/CL_AGROUP+WHO+1.0.xml"><structure:Name xml:lang="en">Age Group</structure:Name></structure:CodeList>

    Note that the agencyID needs to be changed to your agency, matching the one in the OrganisationScheme file.
Hierachical Codelists
  1. In the VOLATILE_CODELIST_HIERARCHY HierachicalCodeList use the CodelistRef element to create aliases to each codelist you wish to create a structure for.
  2. For each hierachy in the HierachicalCodeList create the structures you need using the CodeRef element which represents a node in the structure. Each CodeRef element has a CodelistAliasRef to reference the alias you created and a CodeID which references the codelist code value the alias points to.
Concept Schemes
  1. If you need to create a new Concept that is NOT already defined in the following two files:

    1. COMMON\SDMX-HD\v1.0\conceptschemes\COMMON+SDMX-HD+1.0.xml
    2. COMMON\SDMX-HD\v1.0\conceptschemes\COMMON_GEN+SDMX-HD+1.0.xml

      1. Create a new ConceptScheme element to hold the list of your custom concepts
      2. Add the concepts into the ConceptScheme. Here is an example of a concept called TOPIC, added to the list of custom WHO concepts:

        <structure:ConceptScheme id="CS_CUSTOM" agencyID="WHO" version="1.0" isFinal="false" 
        urn="urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=WHO:CS_CUSTOM">
          <structure:Name xml:lang="en">WHO Custom Concepts</structure:Name>
          <structure:Description xml:lang="en">This Concept Scheme contains 
          un-harmonized data elements.</structure:Description>
          <structure:Concept id="TOPIC" agencyID="WHO" version="1.0"  
          urn="urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=WHO:CS_CUSTOM_WHO[1.0].TOPIC" >
        	<structure:Name xml:lang="en">Health Care Topic.</structure:Name>
        	<structure:Description xml:lang="en">Health Care Topic.</structure:Description>
        	<structure:TextFormat textType="String" />
          </structure:Concept>
          ...
        </structure:ConceptScheme>
        	
Key Family Component Dimensions
  1. Many of the common dimensions you need have been added to the TemplateDSD.xml and commented out. Simply uncomment the dimensions you need.
  2. Should you need to add a new custom dimension, first check the list of existing custom dimensions (meant to be overriden by agency) and uncomment the one you need. Or add a new one if not listed.
  3. Check the following attributes are correct:

    1. conceptSchemeRef
    2. conceptVersion
    3. conceptSchemeAgency
    These attributes need to match the ones specified for your concept

    1. codelist
    2. codelistVersion
    3. codelistAgency
    These attributes need to match the ones specified for your codelist
Groups
No changes needed for indicator definitions

Attributes
No changes needed for indicator definitions

Create your MSD.xml file Codelists
  1. Open the MSD.xml file with the XML editor of your choice.
  2. Update any codelists that have isFinal=false with additional items that you need.
  3. Create any "custom" codelist you need in the "CUSTOM/WHO/v1.0/codelists" folder.
  4. Add these new custom codelist to the MSD file e.g here is custom NEWLIST codelist:

    <structure:CodeList id="CL_NEWLIST" agencyID="WHO" version="1.0" isFinal="false" urn="urn:sdmx:org.sdmx.infomodel.codelist=WHO:CL_NEWLIST" isExternalReference="true" uri="./CUSTOM/WHO/v1.0/codelists/CL_NEWLIST+WHO+1.0.xml"><structure:Name xml:lang="en">NewList</structure:Name></structure:CodeList>

Concept Schemes
  1. If you need to create a new concept that is NOT already defined in the following two files:

    1. COMMON\SDMX-HD\v1.0\conceptschemes\COMMON+MSD+SDMX-HD+1.0.xml
    2. COMMON\SDMX-HD\v1.0\conceptschemes\COMMON_GEN+SDMX-HD+1.0.xml

      1. Create a new ConceptScheme element to hold the list of your custom concepts
      2. Add the concepts into the ConceptScheme. Here is an example of a concept called TEST, added to the list of custom WHO concepts:

        <structure:ConceptScheme id="CS_CUSTOM_MSD_WHO" agencyID="WHO" version="1.0" isFinal="true"
         urn="urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=WHO:CS_CUSTOM_MSD_WHO">
          <structure:Name xml:lang="en">WHO Custom Concepts</structure:Name>
          <structure:Description xml:lang="en">This Concept Scheme contains un-harmonized
           metadata elements.</structure:Description>
          <structure:Concept id="TEST" agencyID="WHO" version="1.0"  
          urn="urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=WHO:CS_CUSTOM_MSD_WHO[1.0].TEST" >
        	<structure:Name xml:lang="en">TEST.</structure:Name>
        	<structure:Description xml:lang="en">TEST</structure:Description>
        	<structure:TextFormat textType="String" />
          </structure:Concept>
          ...
        </structure:ConceptScheme>
        	
IdentifierComponent
No changes needed for indicator definitions

ReportStructure Metadata Attributes
For each metadata attribute linked to the indicator definition (i.e. to the CL_INDICATOR code list), you must create a new MetadataAttribute describing this metadata attribute. Here is an example for Preferred Source (PSOURCE):
<structure:MetadataAttribute conceptRef="PSOURCE" usageStatus="Conditional" 
	conceptSchemeAgency="SDMX-HD" conceptVersion="1.0" conceptSchemeRef="CS_COMMON_MSD"
	representationScheme="CL_PSOURCE" representationSchemeAgency="SDMX-HD">
	<structure:TextFormat textType="String" />
</structure:MetadataAttribute>
					

Create your GMS.xml file AttributeValueSet
  1. Open the GMS.xml file with the XML editor of your choice.
  2. For each indicator metadata row, repeat the following steps:

    1. Locate the AttributeValueSet element and set the ComponentValue value attribute to the IdentifierComponent ID specified in the MSD.
    2. Also change the ? to the codelist value this attributeSets metadata will apply to. So metadata for the first indicator (represented by code 0 in CL_INDICATOR code list), would look like this:
      <gen:TargetValues>
      	<gen:ComponentValue component="IC_INDICATOR" object="Codelist">0</gen:ComponentValue>
      </gen:TargetValues>
    3. Repeat this process for each concept you wish to retain metadata for:
      1. Set the ReportedAttribute conceptID to the concept for which you wish to create metadata for e.g. _CODE, DTYPE etc.
      2. Now set the Value element of the ReportedAttribute to the codelist value for your metadata or the free text value if no codelist exists. String values should be encapsulated into CData tags to avoid XML errors due to string content.
top

Exchanging indicator definitions and data

Setting up the MyWork folder
  1. Identical to "Exchanging indicator definitions" section above
To exchange indicator data, you can start with the same DSD.xml, MSD.xml and GMS.xml that you use to exchange indicator definitions. It is normal that these three files are almost identical as the data structure defining indicators does not change regardless of whether your SDMX-HD message contains indicator data or not. As a matter of fact, only a few changes are required to the DSD.xml. Most of the work involved with including indicator data in the SDMX-HD message consists of constructing the CDS.xml and its accompanying CDS.xsd.

Create your DSD.xml file Codelists
Identical to "Exchanging indicator definitions" section above
Hierachical Codelists
Identical to "Exchanging indicator definitions" section above
Compact Schemes
Identical to "Exchanging indicator definitions" section above
Key Family Component Dimensions
Identical to "Exchanging indicator definitions" section above
Groups
Uncomment the PROGRAMS group element as indicator data recorded in the CDS.xml will be grouped by PROGRAMS.

Attributes
  1. To create new attributes just add the Attribute element and complete the attributes as follows:
    1. conceptSchemeRef - References the ConceptsScheme that contains the concept you wish to use
    2. conceptRef - References the concept for this attribute. The concept ID also become the attribute name used in the CDS.xml file
    3. conceptVersion - The Version of the concept
    4. conceptSchemeAgency - The AgencyID of the concept
    5. attachmentLevel - At which level do you want to attached this attribute : Observation, Series, Group or DataSet
    6. assignmentStatus - "Mandatory" or "Conditional"

Create your MSD.xml file Codelists
Identical to "Exchanging indicator definitions" section above
Compact Schemes
Identical to "Exchanging indicator definitions" section above
IdentifierComponent
Identical to "Exchanging indicator definitions" section above
ReportStructure Metadata Attributes
Identical to "Exchanging indicator definitions" section above

Create your GMS.xml file AttributeValueSet
Identical to "Exchanging indicator definitions" section above

Create your CDS.xsd file CDS Schema Validation File
  1. The details of this process are listed in the SDMX-HD standard documentation in section C2 "XSD schema validation".

Create your CDS.xml file DataSet
  1. The dimensions plus all the attributes listed in the DSD attached to the DataSet must be added to the DataSet elemnent. Add the Mandatory attributes to this level.
Group
  1. PROGRAMS - specify the PROG attribute and set its value to the CL_PROGRAM codelist value to represent the PROGRAM by which you want Series (indicators) to be grouped by.
Series
  1. Each Series represents an indicator. As such, it must have the following attributes defined:
    • PROG - To identify which PROGRAMS group this indicator belongs to
    • INDICATOR - To identify which INDICATOR this Series represent
    • DISEASE - To identify which DISEASE this indicator belongs to
    • GEOGRAPHIC_PLACE_NAME - To identify which Geographical place Observations represent
    • ORGANIZATION - To identify which Organization provided the Observations
    • VALUE_TYPE - To identify the type of observation values for this indicator : Values or Targets
    • PERIODICITY - How often this data should be gathered
    • UNIT - The units of measure for the observation values
    • REPEATS - How many observation values can we have for this indicator. 0 = unlimited.
  2. You then need to add the attributes defining disaggregations specified for this indicator (if any). For exmaple
    • GENDER - The DSD has GENDER specified as dimension, so a value of "_ALL" means all gender
    • AGROUP - The DSD has AGROUP specified as dimension, so a value of "5" means "15+"
    • GLOCATION - The DSD has GLOCATION specified as dimension, so a value of "3" means "Urban"
Observation
  1. Each Observation represents an indicator value and so must have the following attributes set:
    • OBS_VALUE - The indicator data point.
    • TIME_PERIOD - The time period codelist value that defines when this observation value is for.
    • DATE_COLLECT - The date when the data was collected
top

Package the SDMX-HD message as a ZIP archive

  1. The folder structure described above should be respected when packaging files as a ZIP archive. All files described above, including *.xsd and all *.xml files externally referenced by your DSD.xml and MSD.xml must be included in the package.
  2. In effect, just zip the entire "SDMX-HD.v1.0 MyWork" folder contents

    It is suggested to adopt helpful *.zip naming conventions, such as a file name built like this:
     
    SYSTEM_ORGANIZATION_GEOPLACE_YYYYMMDDHHMMSS.zip, where
    • SYSTEM: is the system (or application) creating the SDMX-HD message
    • ORGANIZATION: is the organization reporting the data
    • GEOPLACE: is the geographic place for which data is reported
    • YYYYMMDDHHMMSS represent the file creation timestamp with YYYY= 4-digit year; MM= 2-digit month, etc...
       
top

Avoiding common mistakes

First and foremost, PLEASE VALIDATE YOUR XML FILES PRIOR TO SENDING THEM TO ANOTHER ORGANIZATION. Failure to validate your XML files will likely results in errors that will prevent a successful data exchange as the recipient organization will not be able to import your file. "Stylus Studio 6 XML Professional Edition" and metadata technology SDMX Offline Tools have been successfully used by WHO to validate XML documents against the published schemas.
 
The following bullets points list common mistakes that should be avoided in order to produce compliant XML files: