Non-Normative Guidance in Using NIEM with JSON

Abstract

This document provides guidance from the NIEM Technical Architecture Committee (NTAC) for using the National Information Exchange Model (NIEM) with JavaScript Object Notation (JSON). NIEM provides a well-established standard for defining information exchanges. JSON is specified by RFC4627.

Status

This guidance is non-normative. It discusses possible NIEM conformance rules for JSON data, but it does not establish any specification for such conformance.

This document is a working draft; revisions and additions are expected. Readers are invited to provide feedback on this document by entering an issue at https://github.com/NIEM/NIEM.github.io/issues or sending email to niem-comments@lists.gatech.edu.

Authors

Scott Renner sar@mitre.org, @iamdrscott
Webb Roberts webb.roberts@gtri.gatech.edu, @webb
Leila Tite leila.tite@hennepin.us, @leilatite
Tom Carlson tom@tomcarlsonconsulting.com

Introduction
JSON-LD template for NIEM IEP
JSON-LD representation of NIEM XML
Implementing Translators
Advanced Concerns for JSON-LD Serialization
Additional guidance
- Expressing types
- Linking contexts via HTTP headers
Road map for future work
- NIEM Conformance
- Support for transforming between XML and JSON
References
Full example:
- Full example: XML instance document
- Full example: JSON data
  - RDF

Introduction

This document is provided as guidance, not as a normative specification. It discusses possible NIEM conformance rules for JSON data, but it does not establish any specification for such conformance. This document is the first step in establishing a repeatable and interoperable methodology for using JSON to represent NIEM-conformant information exchanges. The guidelines presented by this document should be in alignment with later normative specifications that define NIEM-conformant JSON.

Audience

The primary target of this guidance is the developer who has a NIEM IEPD that defines one or more XML messages, and who wishes to exchange JSON representations of those messages, instead of XML. For these users, this document supplies patterns for converting an NIEM XML document (i.e., an IEP, an information exchange package) into a semantically-equivalent JSON serialization.

This document also provides guidance for the developer who wishes to use NIEM definitions in a JSON object design. Guidance for developers who wish to map existing JSON objects to NIEM XML may be provided at a later date.

Overview

This document is one step in the NTAC technical road map for JSON, Using JSON with NIEM. It describes NIEM’s methodology for creating consistent, interoperable JSON messages that are based on NIEM-conformant XML schemas. Core aspects of this guidance are:

The guidance describes JSON messages based on NIEM Schemas.
The JSON messages are expressed as JSON-LD, a scalable JSON framework for linked data.
The names, relationships and structures in JSON messages are based on names, relationships and structures described by NIEM-conformant XML schemas.
The details of how JSON-LD represents NIEM XML data is based on the NIEM Naming and Design Rules’s mapping between XML (and XML Schema) and RDF.

The NIEM data model and RDF

The NIEM Naming and Design Rules (NDR) is the main document that explains the meaning of NIEM–conformant XML schemas and XML instance documents. The framework that the NIEM NDR relies on for meaning is the Resource Description Framework (RDF), which defines a data model that is the basis for Semantic Web technologies. NIEM defines the meaning of conformant XML schemas and XML instance documents in terms of RDF. This document leverages the NDR’s RDF to map NIEM XML schemas and XML instance documents to JSON-LD.

Although NIEM is, at its core, defined in terms of RDF, most users of NIEM do not have to understand much about RDF; NIEM is primarily discussed in terms of XML and XML schema: rules about elements, attributes, simple and complex types, etc. Similarly, users of this guidance document, and of NIEM’s JSON-LD representation, will not need to understand much about RDF. The few RDF concepts this document explicitly requires are explained without requiring deep understanding of the underlying RDF concepts.

The NIEM NDR Section 5, “The NIEM conceptual model” describes the conceptual model of NIEM, and provides a mapping between NIEM-conformant XML schemas and XML instance documents, and RDF. This section is the basis for much of what appears in this document. For example:

Section 5.6.1, “Resource IRIs for XML Schema components and information items” describes how to find an internationalized resource identifier (IRI) for a schema component or XML element or attribute, using the namespace name and local name of the component’s qualified name (QName). This is the basis for the translation of element names into JSON-LD terms.
Section 5.6.5.4, “Element as a simple property of an object or association”, describes the RDF for an element of an object or association type. This is the basis for how a NIEM-conformant XML element is translated into corresponding JSON-LD.

These, and other sections of the NDR are the basis of the translation from NIEM XML to JSON-LD. A goal of this document is to explain the translations from NIEM to JSON-LD sufficiently, so that the reader does not need to understand the NDR’s mappings to RDF, nor the JSON-LD’s RDF basis.

Use of JSON-LD

This document describes the translation of NIEM-conformant XML instance documents (informed by certain features of schemas, such as component types) into JSON, using JSON-LD as the target JSON representation. JSON-LD (JavaScript Object Notation for Linking Data) is a language-independent data format for representing Linked Data, expressed as JSON. JSON-LD provides a lightweight mechanism that allows data to be linked across websites. The syntax is easy for humans to read and easy for machines to parse and generate.

One reason for choosing JSON-LD is its context mechanism, which allows IRIs in JSON-LD to look like XML qualified names (QNames). As described above, the NDR defines how to translate element QNames to IRIs. These IRIs can be represented in a short form using a JSON-LD context.

Take, for example, the following JSON data:

{
  "http://release.niem.gov/niem/niem-core/3.0/#quantityUnitText" : "dozen"
}

The key in this JSON object is the IRI defined by NIEM for the nc:quantityUnitText attribute. Using the JSON-LD context mechanism, that IRI can be compacted. as follows:

{
  "@context": {
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#"
  },
  "nc:quantityUnitText": "dozen"
}

The benefits of a JSON-LD representation are apparent. The context object preserves the information in the XML namespace declarations. The JSON object key is a familiar NIEM QName. That QName can be converted into the full IRI by an expansion algorithm defined for JSON-LD by JSON-LD 1.0 Processing Algorithms and API, Section 2.1, “Expansion”.

A second reason for choosing JSON-LD is that JSON-LD is a concrete RDF syntax, like RDF/XML or Turtle. This means that NIEM JSON-LD data may be processed using RDF techniques, such as SPARQL queries. This also means that there are now two methods of generating RDF from NIEM-conforming XML documents and schemas: the first defined in NIEM NDR Section 5, the second described by this guidance. The NTAC’s intention is that the two methods produce consistent RDF; realizing this intention may require future changes to the NDR, this guidance, or both.

Based on the above, all three of these methods are legitimate ways to access NIEM JSON-LD data:

A developer can work with the data as if it is plain JSON, without worrying about details of JSON-LD.
A developer can use JSON-LD tools as part of the data processing, including using compaction against a known context, expansion, flattening, or framing.
A developer can process the data as RDF, possibly first converting it to RDF/XML or Turtle syntax.

The guidance in the next major section is intended to describe a JSON-LD serialization of NIEM-conforming XML that is easy to generate, and convenient to process, for all three of these consumer use cases. Certain advanced concerns for that serialization may be found in section 4. This document does not attempt to teach best practices for JSON or JSON-LD; for that, the reader may turn to http://json-ld.org/spec/latest/json-ld-api-best-practices, or other similar resources.

NIEM XML and NIEM JSON carry the same data

A NIEM-conformant schema defines a simple object model. Here is an example of data that is defined by a NIEM schema:

object model

This data can be represented in either of two forms: XML or JSON. Here are an XML and a JSON representation for this data, side-by-side:

<?xml version="1.0" encoding="UTF-8"?>                                  {
<cyfs:PersonOtherKinAssociation                                           "@context" : {
  xmlns:cyfs="http://release.niem.gov/niem/domains/cyfs/3.2/"               "cyfs": "http://release.niem.gov/niem/domains/cyfs/3.2/#",
  xmlns:j="http://release.niem.gov/niem/domains/jxdm/5.2/"                  "j": "http://release.niem.gov/niem/domains/jxdm/5.2/#",
  xmlns:nc="http://release.niem.gov/niem/niem-core/3.0/">                   "nc": "http://release.niem.gov/niem/niem-core/3.0/#"
                                                                          },
                                                                          "cyfs:PersonOtherKinAssociation": {
  <cyfs:SourcePerson>                                                         "cyfs:SourcePerson": {
    <nc:PersonAgeMeasure>                                                     "nc:PersonAgeMeasure": {
      <nc:MeasureIntegerValue>14</nc:MeasureIntegerValue>                       "nc:MeasureIntegerValue": 14,
      <nc:TimeUnitCode>ANN</nc:TimeUnitCode>                                    "nc:TimeUnitCode": "ANN"
    </nc:PersonAgeMeasure>                                                    },
    <j:PersonHairColorCode>BRO</j:PersonHairColorCode>                        "j:PersonHairColorCode": "BRO",
    <nc:PersonName>                                                           "nc:PersonName": {
      <nc:PersonGivenName>Mortimer</nc:PersonGivenName>                         "nc:PersonGivenName": "Mortimer",
      <nc:PersonSurName>Smith</nc:PersonSurName>                                "nc:PersonSurName": "Smith",
      <nc:PersonNameSuffixText>Sr</nc:PersonNameSuffixText>                     "nc:PersonNameSuffixText": "Sr",
      <nc:PersonPreferredName>Morty</nc:PersonPreferredName>                    "nc:PersonPreferredName": "Morty"
    </nc:PersonName>                                                          }
  </cyfs:SourcePerson>                                                      },
  <cyfs:TargetPerson>                                                       "cyfs:TargetPerson": {
    <j:PersonHairColorCode>GRY</j:PersonHairColorCode>                        "j:PersonHairColorCode": "GRY",
    <nc:PersonName>                                                           "nc:PersonName": {
      <nc:PersonGivenName>Rick</nc:PersonGivenName>                             "nc:PersonGivenName": "Rick",
      <nc:PersonSurName>Sanchez</nc:PersonSurName>                              "nc:PersonSurName": "Sanchez"
    </nc:PersonName>                                                          }
  </cyfs:TargetPerson>                                                      },
  <cyfs:PersonOtherKinAssociationCategoryCode                               "cyfs:PersonOtherKinAssociationCategoryCode":
    >Maternal Grandfather</cyfs:PersonOtherKinAssociationCategoryCode>        "Maternal Grandfather"
</cyfs:PersonOtherKinAssociation>                                         }
                                                                        }

You can exchange NIEM data either as XML or as JSON. Either way, the data that is expected in a message, and the meaning of that data, is defined by NIEM schemas.

The NIEM JSON representation uses JSON-LD, which means that order of NIEM JSON data is flexible, that the names of keys is determined by JSON-LD contexts, that the JSON representation is also an RDF representation, and that there are additional tools available for working with NIEM JSON.

One method of facilitating NIEM JSON exchanges is for someone who understands NIEM schemas to construct a template JSON instance, which is a NIEM JSON instance for a NIEM schema, but with data values replaced with null. This method is described by the next section.

Another method of facilitating NIEM JSON exchanges is to translate between NIEM XML and NIEM JSON. This is described in a section below. That section also describes how NIEM JSON represents data that is more complex. Although NIEM XML is used in that section describe how NIEM JSON works in various situations, you do not have to work with NIEM XML or translate between JSON and XML to work with NIEM JSON.

JSON-LD template for NIEM IEP

One way to make it simpler for the JSON developer to create JSON-LD documents for a NIEM exchange is to provide a fully completed JSON-LD document structure with all “null” values. The IEP designer provides an iep-sample-template.jsonld document in the iep-samples directory that describes the context and structure of the document for the exchange. The developer populates the fields for which they have values and submits the document. All elements with null values are ignored during processing at the receiving end.

The following JSON example is a JSON IEP template for the simple object model from Section 1.5, above.

{
  "@context": {
    "exch": "http://example.com/SimpleExchange/1.0/#",
    "j": "http://release.niem.gov/niem/domains/jxdm/5.1/#",
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#",  
    "cyfs": "http://release.niem.gov/niem/domains/cyfs/3.2/"
  },
  "cyfs:PersonOtherKinAssociation": {
    "cyfs:SourcePerson": {
      "nc:PersonAgeMeasure": {
        "nc:MeasureIntegerValue": null,
        "nc:TimeUnitCode": null
      },
      "j:PersonHairColorCode": null,
      "nc:PersonName": {
        "nc:PersonGivenName": null,
        "nc:PersonSurName": null,
        "nc:PersonNameSuffixText": null,
        "nc:PersonPreferredName": null
      }
    },
    "cyfs:TargetPerson": {
      "j:PersonHairColorCode": null,
      "nc:PersonName": {
        "nc:PersonGivenName": null,
        "nc:PersonSurName": null
      }
    },
    "cyfs:PersonOtherKinAssociationCategoryCode": null
  }
}

JSON-LD representation of NIEM XML

This section describes the translation of an example IEP (information exchange package, an XML instance document), defined by an IEPD (information exchange package description), into the corresponding JSON-LD data. Each section within highlights the transformation of a NIEM or XML concept into corresponding JSON-LD. The full source XML appears in an appendix below. The resulting JSON-LD also appears in an appendix below.

This section makes simplifying assumptions, which may not apply to every NIEM IEP. If your IEP is more complicated, then you may have to extend the guidelines to cover your data.

Throughout these examples, at each stage, content is omitted or held back for later sections. When content is left out, an ellipsis (“…”) appears in place of the omitted text.

IEP XML instance document

The IEP as a whole is represented by a JSON object, evident in an instance by an outside set of curly braces. The sub-objects for the root element and @context go within that object:

{
    "@context" : {
        ...
    },
   ...
}

Rules for what goes in this object follow below.

Namespaces and JSON-LD context

NIEM uses XML namespaces to distinguish components with similar names, to identify the authorities responsible for managing a set of XML schema components, and to organize data definitions among NIEM Core, domains, IEPD extensions, etc. Every NIEM IEP has namespace declarations for its content. To uniquely specify a NIEM element, attribute, or schema component, its local name must be combined with the namespace associated with its namespace prefix.

The context mechanism in JSON-LD can serve a purpose similar to XML namespaces. JSON-LD uses International Resource Identifiers (IRIs) as keys within JSON objects, and to uniquely identify objects. A JSON-LD context can define parts of IRIs, which can be used in short-form terms which expand into full IRIs. So, XML namespaces in the IEP are mapped to JSON-LD context entries, to support short-form terms.

ASSUMPTIONS: All of the IEP’s namespace declarations are at or above the IEP’s root element (if there is an envelope around the IEP’s root element). There is no content using the default (no namespace prefix) namespace.

These are the namespace declarations in the sample IEP:

<exch:CrashDriverInfo
    xmlns:exch="http://example.com/CrashDriver/1.0/"
    xmlns:j="http://release.niem.gov/niem/domains/jxdm/5.1/"
    xmlns:nc="http://release.niem.gov/niem/niem-core/3.0/"
    xmlns:geo="http://release.niem.gov/niem/adapters/geospatial/3.0/"
    xmlns:gml="http://www.opengis.net/gml/3.2"
    xmlns:structures="http://release.niem.gov/niem/structures/3.0/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <nc:Person structures:id="P01">
  ...
</exch:CrashDriverInfo>

Each declared namespace prefix is converted into an @context entry:

The namespace prefix becomes a short term.
The namespace name assigned to the prefix is converted into an IRI root.

The rules for constructing IRIs from QNames are provided by NDR section 5.6.1, “Resource IRIs for XML Schema components and information items”. This leads to the definition of IRI roots as follows:

If the namespace name ends with “#”, then the IRI root is the namespace name.
Otherwise: concatenate(namespace name, “#”).

For the QName nc:Person this yields the IRI http://release.niem.gov/niem/niem-core/3.0/#Person.

In addition, we apply the following guidelines:

Omit the xsi namespace, http://www.w3.org/2001/XMLSchema-instance, which is reserved for XML Schema-specific concepts, and is not carried over into JSON-LD.
Include a declaration of rdf to http://www.w3.org/1999/02/22-rdf-syntax-ns#. This is used for rdf:value and rdf:XMLLiteral.

This yields the following @context entry:

{
    "@context" : {
        "exch" : "http://example.com/CrashDriver/1.0/#",
        "j" : "http://release.niem.gov/niem/domains/jxdm/5.1/#",
        "nc" : "http://release.niem.gov/niem/niem-core/3.0/#",
        "geo" : "http://release.niem.gov/niem/adapters/geospatial/3.0/",
        "gml" : "http://www.opengis.net/gml/3.2#",
        "structures" : "http://release.niem.gov/niem/structures/3.0/#",
        "rdf" : "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    },
    "exch:CrashDriverInfo" : {
        ...
    }
}

Document element (A.K.A. root element)

Each element within the IEP is converted to a key in a JSON-LD node object.

The root element (or document element) of a NIEM IEP is also converted to a node object. It is given the JSON key corresponding to the QName (XML qualified name) of the element. This may be expanded into an IRI by a JSON-LD processor, but this guidance recommends using the QName, supported by a context entry for the IRI root. Within the full example, this yields:

{
  "@context": {
    "exch": "http://example.com/CrashDriver/1.0/#",
    ...
  },
  "exch:CrashDriverInfo": {
    ...
  }
}

Seems to me we need only one of sections 2.1 and 2.3. I’d keep 2.3. —@iamdrscott

Element content of an object or association type

Since the top level element is a NIEM object type, we convert most elements within to a key for the top-level element’s JSON-LD object. In the sample instance, exch:CrashDriverInfo has the following child elements:

nc:Person
j:Crash
j:PersonChargeAssociation
j:Charge
j:JusticeMetadata

These are each converted into keys within the exch:CrashDriverInfo object described above:

...
"exch:CrashDriverInfo": {
  "nc:Person": {
      ...
  },
  "j:CrashDriver": {
      ...
  },
  "j:PersonChargeAssociation": {
      ...
  },
  "j:Charge": {
      ...
  }
  "j:JusticeMetadata": {
      ...
  }
}

This is the most common case; this is how object types and association type are translated to JSON-LD.

Note that there is only one pair for each child element name, no matter how many times that element appears in the content. It is now time to discuss…

Repeated Elements

In the full example XML IEP, The element nc:PersonMiddleName is repeated within nc:PersonName:

<nc:PersonName nc:personNameCommentText="copied">
    <nc:PersonGivenName nc:sequenceID="1">Peter</nc:PersonGivenName>
    <nc:PersonMiddleName nc:sequenceID="2">Death</nc:PersonMiddleName>
    <nc:PersonMiddleName nc:sequenceID="3">Bredon</nc:PersonMiddleName>
    <nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>

Since nc:PersonMiddleName is repeated, it is represented using the key nc:PersonMiddleName, with a value that is an array of objects:

"nc:PersonName": {
  "nc:PersonGivenName": <!-- content of element nc:PersonGivenName -->,
  "nc:PersonMiddleName": [
    <!-- content of first element nc:PersonGivenName -->,
    <!-- content of second element nc:PersonGivenName -->
  ],
  "nc:PersonSurName": <!-- content of element nc:PersonSurName -->,
  "nc:personNameCommentText": "copied"
}

We see that occurrences of element nc:PersonMiddleName are bundled together as an array, and that array is the value for key nc:PersonMiddleName. A repeated element is converted into a JSON object with an array value. (This is done for elements with simple content as well as the complex elements shown.) The array contains one JSON object for the content of each element instance. Note that order of data within an array should not be considered significant, as described below.

Consumers of NIEM JSON-LD data must be aware that the value of a repeatable element may be an array (if the element is in fact repeated), or an object (if it is not), and code accordingly.

Observe that with this guidance, the same JSON is produced for these two Parent elements:

|<Parent>          |    <Parent>          |
|    <Repeated/>   |        <Repeated/>   |
|    <Repeated/>   |        <Child/>      |
|    <Child/>      |        <Repeated/>   |
|</Parent>         |    </Parent>         |

Therefore it is up to the developer to ensure that these two elements have the same meaning in the IEP. If the meaning depends on the difference in element ordering, then the guidance in this document does not apply, and the developer is on his own. However, in that case, the IEPD design is bad, conflicting with the NIEM Conceptual Model.

Element with Complex Content and Attributes

The content of an element with complex content is converted to a JSON object with one pair for the name of each attribute and one pair for the name of each child element in that content. For example, in the sample IEP we have

<nc:PersonName nc:personNameCommentText="copied">
    <nc:PersonGivenName nc:sequenceID="1">Peter</nc:PersonGivenName>
    <nc:PersonMiddleName nc:sequenceID="2">Death</nc:PersonMiddleName>
    <nc:PersonMiddleName nc:sequenceID="3">Bredon</nc:PersonMiddleName>
    <nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>

which is converted to the following JSON

"nc:PersonName": {
  "nc:personNameCommentText": "copied",
  "nc:PersonGivenName": {
    ...
  },
  "nc:PersonMiddleName": [
    ...
  ],
  "nc:PersonSurName": {
    ...
  }
}

Observe that for a NIEM-conforming IEP, there can’t be a collision between attribute and child element names, because of the camel-case rule. For an IEP that is not NIEM conforming, it is possible to have an attribute and a child element with the same name. This document has no guidance for such non-conforming IEPs; developers are on their own.

Element with Simple Content and Attributes

The nc:PersonGivenName element has simple content and attributes. Its JSON representation is

  "nc:PersonGivenName" : {
    "nc:sequenceID" : 1,
    "rdf:value" : "Peter"
  }

The element’s simple content can’t be represented by {"nc:PersonGivenName" : "Peter" }, because then there is no place to put the attributes. Instead, the representation must be an object, with ordinary keys for the attributes and a special key for the simple content. That special key could have a magic syntax, such as ".", but this would only work for plain JSON consumers; to process the data as JSON-LD or RDF, the key must be a term mapped to an IRI.

According to the RDF Primer (2004), rdf:value is customarily used in this situation where a property has one main value and one or more additional values providing context that qualifies the main value. Therefore, this guidance chooses rdf:value for the special key. This choice also makes the JSON-LD representation consistent with NDR content of an element, which specifies that non-empty simple values are mapped in this way.

Obviously this will break if an element in the IEP has rdf:value as an attribute. Fortunately, there is no good reason to do that in a NIEM IEP.

Element with Simple Content and No Attributes

The nc:Date element has simple content and no attributes. Its JSON-LD representation is

  "nc:Date" :"1893-05-04"

With no attributes, there is no need for the rdf:value key, and so it is not used. Consumers of NIEM JSON-LD data must be aware that the value of a simple element may be an object (if the element has attributes), or a value (if it does not), and code accordingly.

Element with Numeric or Boolean Content

When the IEPD schema defines a numeric type for a simple element, the value of the JSON pair is a number. Likewise, when the schema defines a boolean type, the value of the JSON pair is true or false. For example, the representations of nc:MeasureDecimalValue and exch:PersonFictionalCharacterIndicator are:

  "nc:MeasureDecimalValue" : 9.7

  "exch:PersonFictionalCharacterIndicator" : true

Elements with empty and nilled content

There are three situations in an XML instance where an element may have empty content:

The element has no simple content, and any element content is optional. For example, its type may be a complex type with complex content, which has no simple value, and its element children may have minOccurs="0".
The element is defined to have simple or complex content, but the simple content is nilled using xsi:nil="true".
The element has simple content, but that simple content is the empty string.

Cases 1 and 2 are represented the same way in JSON-LD: There is no value associated with the element. Take this XML example (case 1):

<nc:Person/>

…or the equivalent (case 1):

<nc:Person></nc:Person>

Both of these omit any content. To be a valid XML instance, the schema for this must either define nc:PersonType (the type defined by NIEM for element nc:Person) with no child elements, or with child elements that have minOccurs="0". A similar instance may use xsi:nil (case 2):

<nc:Person xsi:nil="true"/>

This XML instance may be valid with mandatory element children, as long as nillable="true" is set for the element. The JSON for all of these is the same:

"nc:Person" : { }

This expresses that nc:Person is a node object, but does not assert any other properties.

An element carrying an empty string (case 2) is represented differently. In NIEM, element nc:PersonGivenName is defined to have simple content based on xs:string. So, if that element is empty, it represents the empty string. The XML instance (case 2):

<nc:PersonGivenName></nc:PersonGivenName>

…and the equivalent XML (case 2):

<nc:PersonGivenName/>

…are both carrying the empty string as children of nc:PersonGivenName. This may be represented with the JSON:

"nc:PersonGivenName" : ""

ID Attributes

NIEM defines the attribute structures:id to carry ID values. structures:id is the only ID-typed attribute allowed in NIEM-conformant content (except for externally-defined content). An ID attribute in XML defines a unique document-relative unique identifier: An ID value may appear in at most one element within a single XML document; the same ID value may appear in any number of different documents.

The NIEM-defined structures:id attribute is represented by the JSON-LD reserved key @id. The value for @id is the value of structures:id. For example, the XML:

<nc:Person structures:id="P01">
  <nc:PersonBirthDate>
    <nc:Date>1893-05-04</nc:Date>
  </nc:PersonBirthDate>
  <nc:PersonName nc:personNameCommentText="copied">
    ...
  </nc:PersonName>
  <exch:PersonFictionalCharacterIndicator>true</exch:PersonFictionalCharacterIndicator>
</nc:Person>

…is represented by the JSON-LD:

"nc:Person" : {
  "@id" : "P01",
  "nc:PersonBirthDate": {
    "nc:Date": "1893-05-04"
  },
  "nc:PersonName": {
    "nc:personNameCommentText" : "copied",
    ...
  },
  "exch:PersonFictionalCharacterIndicator": true
}

The JSON-LD processor will process @id values against a base IRI. This might be automatically generated by a system (for example, the JSON-LD Playground uses a base IRI of http://json-ld.org/playground/). The base IRI may also be set by @base within an @context, as described by JSON-LD 1.0, Section 6.1, “Base IRI”.

References and IDREF attributes

The value of a structures:ref attribute and any other IDREF attribute is converted into a reference using the @id of the corresponding JSON-LD node. For example, the representation for <nc:RoleOfPerson structures:ref="P01" xsi:nil="true"/> is

"nc:RoleOfPerson" : {
  "@id" : "P01"
}

The representation of a reference element has no content from the element value, either complex or simple; that is, the only content is the "@id":"idref" pair and zero or more attributes. In NIEM JSON-LD data created according to this guidance, this will be true of every object representing a reference element. In addition, there must be exactly one object containing the "@id":"value" pair that does have child elements or simple element content. That will be the object representing the element with the structures:id attribute, as described above. These constraints are not part of JSON-LD; it is created by this guidance.

Observe that the xsi:nil attribute, which is useful only for schema validation, does not appear in the JSON-LD representation.

Abstract Elements and Substitution Groups

Because this guidance does not seek to replicate in JSON the underlying XML Schema powering NIEM, substitution groups are simply represented by replicating the XML instance structure, where the substitution has already taken place. Abstract elements do not appear at all. For example,

<nc:Person>
    <!-- Date substituted for DateRepresentation -->
    <nc:PersonBirthDate>
        <nc:Date>1893-05-04</nc:Date>
    </nc:PersonBirthDate>
</nc:Person>

becomes

"nc:Person" : {
    "nc:PersonBirthDate" : {
        "nc:Date"  : "1893-05-04"
    }
}

Augmentations

NIEM has two kinds of augmentation elements. The most common is a container element that is derived from AugmentationType and has a name ending in Augmentation; for example, LicenseAugmentation in the sample IEP. These augmentation container elements are important for schema validation, but have no role in the conceptual model, and so they do not appear in the JSON representation. So, for example, the representation of j:DriverLicense is

"j:DriverLicense" : {
  "j:DriverLicenseCardIdentification" : {
      "nc:IdentificationID" : "A1234567"
  },
  "nc:ItemLengthMeasure" : {
      "nc:MeasureDecimalValue" : 9.7,
      "nc:LengthUnitCode" : "CMT"
  }
}

The other kind of augmentation element is an element declared in the substitution group of an AugmentationPoint. For example, the IEPD schema includes the following element declaration:

<xs:element name="PersonFictionalCharacterIndicator" type="niem-xs:boolean"
  substitutionGroup="nc:PersonAugmentationPoint">
    <xs:annotation><xs:documentation>
        True if this person is a fictional character in a literary work.
    </xs:documentation></xs:annotation>
</xs:element>

Augmentation elements of this kind are treated the same as any other child element.

Metadata

We are still working on figuring out how to represent metadata in JSON-LD. The NDR says to hang metadata on RDF statements, but it is not clear what would be the best way to do that. One option is if the XML contains metadata, then create an RDF statement and hang the metadata off of that. Metadata on objects can be handled with rdf:value, but metadata on associations is harder. —@webb, —@leilatite

One way to do that is through reification. The JSON-LD would look something like example from stack overflow:

{
  "@context": {
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "subject": { "@id": "rdf:subject", "@type": "@id" },
    "predicate": { "@id": "rdf:predicate", "@type": "@id" },
    "object": { "@id": "rdf:object", "@type": "@id" },
    "j": "http://release.niem.gov/niem/domains/jxdm/5.1/#",
    "structures": "http://release.niem.gov/niem/structures/3.0/#"
  },
  "@type": "rdf:Statement",
  "subject": "j:Charge",
  "predicate": "structures:metadata",
  "object": { "@id": "j:JusticeMetadata" },
  "j:CriminalInformationIndicator": true
}

Adapter Elements

IEPD developers sometimes want to reuse schema components defined in an existing standard via a schema that does not conform to the NDR. The adapter elements defined in NDR section 10.2.3, “External adapter types and external components” are the NIEM mechanism for including these external components in the IEPD. For instance, in the following section of the sample IEP, the geo:LocationGeospatialPoint element is a NIEM-conforming adapter. The external content of that adapter element (gml:Point) is defined by the Open Geospatial Consortium (OGC) in the Geographic Markup Language (GML).

<geo:LocationGeospatialPoint>
    <gml:Point gml:id="PT01" srsName="urn:ogc:def:crs:EPSG::4326">
        <gml:pos>51.835 -0.417</gml:pos>
    </gml:Point>
</geo:LocationGeospatialPoint>

Because the external content of an adapter element does not follow the NDR, the guidance in this document may not apply. Developers must decide how to convert external content to JSON on a case-by-case basis, and must also decide which of the three consumer use cases (plain JSON, JSON-LD, RDF) they will support. For the adapter element in the sample IEP, the developer might apply any of the following solutions:

Pretend the external content is NIEM conforming

The developer could simply apply the guidance in this document as if the external content were NIEM conforming. In this case, the adapter element is converted to

"geo:LocationGeospatialPoint": {
  "gml:Point": {
    "@id": "PT01",
    "srsName": "urn:ogc:def:crs:EPSG::4326",
    "gml:pos": "51.835 -0.417"
  }
}

That will work when processed as plain JSON. When processed as JSON-LD, the resulting IRIs in the expanded form are reasonable. However, because the srsName attribute does not have a namespace in GML, it will be dropped by the JSON-LD processor. That will probably cause difficulties for the JSON-LD and RDF consumer use cases.

Create a custom mapping to JSON-LD

The difficulty with the srsName attribute could be resolved with a custom mapping, one which extends the guidance in this document by supplying a special @context pair for the content of the adapter element. The resulting JSON-LD would be

"geo:LocationGeospatialPoint": {
  "@context": {
    "srsName": "http://www.opengis.net/gml/3.2#srsName"
  },
  "gml:Point": {
    "@id": "PT01",
    "srsName": "urn:ogc:def:crs:EPSG::4326",
    "gml:pos": "51.835 -0.417"
  }
}

This approach produces a plausible IRI for the srsName attribute in the expanded JSON-LD, and so should work for the JSON-LD and RDF consumer use cases. This is the approach followed for the full example

Represent the external content with GeoJSON

GeoJSON is a geospatial data interchange format based on JSON. The Internet Engineering Task Force (IETF), in conjunction with the original specification authors, has formed the Geographic JSON WG to standardize the format. Although GeoJSON is not a specification of the OGC, it is still a plausible choice for encoding GML content in JSON. With this solution, the adapter element is converted to

"geo:LocationGeospatialPoint": {
  "@context": {
    "geometry": "https://datatracker.ietf.org/doc/draft-ietf-geojson/#geometry",
    "type": "https://datatracker.ietf.org/doc/draft-ietf-geojson/#type",
    "coordinates": "https://datatracker.ietf.org/doc/draft-ietf-geojson/#coordinates"
  },
  "geometry": {
    "type": "Point",
    "coordinates": [51.835, -0.417]
  }
}

That will work when processed as plain JSON, ignoring the @context pair. Defining the context as shown will produce plausible IRIs in the expanded JSON-LD and converted RDF. A better base IRI would be desirable, of course.

Represent the external content as an XMLLiteral blob

"geo:LocationGeospatialPoint": {
  "@type": "rdf:XMLLiteral",
  "@value":
    "<gml:Point gml:id=\"PT01\"
                srsName=\"urn:ogc:def:crs:EPSG::4326\">
         <gml:pos>51.835 -0.417</gml:pos>
     </gml:Point>"
}

Implementing Translators

TODO: Discuss an easy button transform from NIEM XML to JSON-LD, treating input like a canned query with very little optionality, transforming to JSON-LD using XSLT3’s JSON capability. —@webb

Advanced Concerns for JSON-LD Serialization

JSON-LD as plain JSON

A software developer may wish to work with JSON-LD data instances with vanilla JSON tools, which aren’t JSON-LD aware. Although developers are encouraged to use JSON-LD tools, using vanilla JSON tools is straightforward, although there are caveats. These include:

The meaning of JSON-LD is defined its context.
JSON-LD expanded syntax may be much more regular, simplifying software that uses data.
The order of keys in an object is not significant, so don’t rely on it.
Arrays may only appear as needed.
Order of data inside an array should not be considered significant (as described by Building JSON-LD APIs: Best Practices, “Best Practice 6: Assume arrays are unordered”)

Vanilla JSON processes that use JSON-LD are encouraged to carefully control the organization and JSON-LD context of the data. This may be done by using the expansion, compaction, and framing algorithms provided as part of the JSON-LD specification products.

Order of keys not significant

In JSON and JSON-LD, the order of keys within an object are not considered significant. This means that the two following pieces of JSON-LD are equivalent:

{
  "ns:key1": "value",
  "ns:key2": "value"
}

is equivalent to:

{
  "ns:key2": "value",
  "ns:key1": "value"
}

The order of keys within a JSON-LD instance may be determined by the writer of the JSON-LD, or may be handled automatically by a JSON-LD library.

Arrays may be omitted

In JSON-LD, a single object is equivalent to an array that contains a single object. So, the two following pieces of JSON-LD are equivalent:

{
  "ns:key": "value"
}

is equivalent to:

{
  "ns:key" : [
    "value"
  ]
}

A system that is generating JSON-LD could optionally generate the second form instead of the first.

Additional guidance

Expressing types

The @type keyword is used to associate a type with a node. The concept of a node type and a value type are different. A node type specifies the type of thing that is being described, like a Person, Location, or Event. A value type specifies the data type of a particular value, such as an integer, a floating point number, or a date. Using @type keywords is optional. In general, the NIEM IEPD uses XML Schema to define node types or to constrain value types so it would not be necessary for the JSON-LD to repeat all of that.

If you need to specify type information for handling the iep data correctly, you can add it where needed. This example specifies that the node type of Person in the iep is a NIEM PersonType and the value type of their PersonBirthDate is an xsd date type.

{
  "@context": {
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#",
    "xs": "http://www.w3.org/2001/XMLSchema#",
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    },
      "nc:Person": {
        "@id": "P01",
        "@type" : "nc:PersonType",
        "nc:PersonBirthDate": {
          "nc:Date": {
            "rdf:value": {
              "@value" : "1893-05-04",
              "@type"  : "xs:date"
            }
          }
        }
      }
    }

Here is an example of a typed literal, identifying the type of the simple content of nc:MeasureDecimalValue as being type xs:decimal:

{
  "@context" : {
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#",
    "xs": "http://www.w3.org/2001/XMLSchema#"
  },
  "nc:MeasureDecimalValue": {
    "rdf:value": {
      "@value": "9.7",
      "@type" : "xs:decimal"
    }
  }
}

If you don’t specify the value type as xs:decimal, it is interpreted by JSON as being of type http://www.w3.org/2001/XMLSchema#double and the value becomes “9.699999999999999E0”.

See JSON-LD Section 6.4 Typed Values for more information on how to express typed values.

Linking contexts via HTTP headers

If you have existing JSON data that you want to expose as JSON-LD, you can do that by supplying a separate JSON-LD context for it. This provides an upgrade path for developers who need to be able to continue to support regular JSON. You do this by adding a HTTP Link Header as specified by RFC5988 using the http://www.w3.org/ns/json-ld#context link relation.

A separate context file could be served at URL http://example.com/contexts/iepd-context.jsonld:

{
  "@context": {
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#",
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    ...
    "givenName" : "nc:PersonGivenName",
    "additionalName" : "nc:PersonMiddleName",
    "familyName" : "nc:PersonSurName",
    "text" : "rdf:value"
  }
}

A link header can be added to the plain JSON response that includes:

Link: <http://example.com/contexts/iepd-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"

The JSON-LD processor prepends the context to the JSON data and interprets the original plain JSON as JSON-LD. These definitions can then be used by a JSON data instance, without the instance explicitly referring to the context. Since the context defined the short terms as aliases to full NIEM terms, expanding this JSON data will convert it to use the full NIEM terms.

{
  "givenName": { "text" : "Peter" },
  "additionalName": [ { "text": "Death"},
                      { "text" : "Bredon" } ],
  "familyName": { "text" : "Wimsey" }
}

A benefit of separating the context from the JSON is that if you’re storing the JSON data in a database you can add new terms to the JSON-LD context without having to replace the inline context in all the data that has already been stored.

See JSON-LD Specification Section 6.8, “Interpreting JSON as JSON-LD” for more information and examples of how to implement this.

Road map for future work

NIEM Conformance

Provide a normative specification on a way to generate a JSON-LD instance from a NIEM-conformant XML schema and/or XML instance, and such a JSON-LD instance could be NIEM-conformant.
Provide a normative specification on verifying NIEM-conformance of a JSON-LD instance to a NIEM-conformant XML schema

Support for transforming between XML and JSON

Describe approaches or tools for transforming.

Provide helper functions to check for repeatable elements.

References

JSON-LD: Manu Sporny, Gregg Kellogg, Markus Lanthaler, Editors. 16 January 2014. W3C Recommendation. “A JSON-based Serialization for Linked Data.” Available from https://www.w3.org/TR/json-ld/
RDF-Concepts: Richard Cyganiak, David Wood, Markus Lanthaler, Editors. 09 January 2014. W3C Proposed Recommendation. “RDF 1.1 Concepts and Abstract Syntax.” Available from http://www.w3.org/TR/rdf11-concepts/
RDF Primer: Frank Minola, Eric Miller, Brian McBride, Editors. 10 February 2004. W3C Recommendation. 10 February 2004. “RDF Primer.”
RFC4627: D. Crockford. “The application/json Media Type for JavaScript Object Notation (JSON) (RFC 4627).” July 2006. RFC. Available from http://www.ietf.org/rfc/rfc4627.txt
RFC5988: M. Nottingham. Web Linking. October 2010. Internet RFC 5988. Available from: http://www.ietf.org/rfc/rfc5988.txt

Full example:

The walk-through of how to transform an XML instance document IEP into a JSON-LD data instance is demonstrated using the full example documents within this section. These examples were constructed to show many aspects and features of NIEM XML instance documents.

Full example: XML instance document

The following XML document is the XML form for the full example from Section 2, above.

<?xml version="1.0" encoding="UTF-8"?>
<exch:CrashDriverInfo 
 xmlns:exch="http://example.com/CrashDriver/1.0/"
 xmlns:j="http://release.niem.gov/niem/domains/jxdm/5.1/"
 xmlns:nc="http://release.niem.gov/niem/niem-core/3.0/"
 xmlns:geo="http://release.niem.gov/niem/adapters/geospatial/3.0/"
 xmlns:gml="http://www.opengis.net/gml/3.2"
 xmlns:structures="http://release.niem.gov/niem/structures/3.0/"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <nc:Person structures:id="P01">
        <nc:PersonBirthDate>
            <nc:Date>1893-05-04</nc:Date>
        </nc:PersonBirthDate>
        <nc:PersonName nc:personNameCommentText="copied">
            <nc:PersonGivenName nc:sequenceID="1">Peter</nc:PersonGivenName>
            <nc:PersonMiddleName nc:sequenceID="2">Death</nc:PersonMiddleName>
            <nc:PersonMiddleName nc:sequenceID="3">Bredon</nc:PersonMiddleName>
            <nc:PersonSurName>Wimsey</nc:PersonSurName>
        </nc:PersonName>
        <exch:PersonFictionalCharacterIndicator>true</exch:PersonFictionalCharacterIndicator>
    </nc:Person>
    <j:Crash>
        <nc:IncidentLocation>
            <geo:LocationGeospatialPoint>
                <gml:Point gml:id="PT01" srsName="urn:ogc:def:crs:EPSG::4326">
                    <gml:pos>51.835 -0.417</gml:pos>
                </gml:Point>
            </geo:LocationGeospatialPoint>
        </nc:IncidentLocation>
        <j:CrashVehicle>
            <j:CrashDriver>
                <nc:RoleOfPerson structures:ref="P01" xsi:nil="true"/>
                <j:DriverLicense>
                    <j:DriverLicenseCardIdentification>
                        <nc:IdentificationID>A1234567</nc:IdentificationID>
                    </j:DriverLicenseCardIdentification>
                    <exch:LicenseAugmentation>
                        <nc:ItemLengthMeasure>
                            <nc:MeasureDecimalValue>9.7</nc:MeasureDecimalValue>
                            <nc:LengthUnitCode>CMT</nc:LengthUnitCode>
                        </nc:ItemLengthMeasure>
                    </exch:LicenseAugmentation>
                </j:DriverLicense>
            </j:CrashDriver>
        </j:CrashVehicle>
    </j:Crash>
    <j:PersonChargeAssociation>
        <nc:Person structures:ref="P01" xsi:nil="true"/>
        <j:Charge structures:ref="CH01" xsi:nil="true"/>
        <j:JuvenileAsAdultIndicator>true</j:JuvenileAsAdultIndicator>
    </j:PersonChargeAssociation>
    <j:Charge structures:id="CH01" structures:metadata="MD01">
        <j:ChargeDescriptionText>Wild Driving</j:ChargeDescriptionText>
        <j:ChargeFelonyIndicator>false</j:ChargeFelonyIndicator>
    </j:Charge>
    <j:JusticeMetadata structures:id="MD01">
        <j:CriminalInformationIndicator>true</j:CriminalInformationIndicator>
    </j:JusticeMetadata>
</exch:CrashDriverInfo>

Full example: JSON data

The following JSON data is a compact JSON-LD form of the full example from Section 2, above.

{
  "@context": {
    "exch": "http://example.com/CrashDriver/1.0/#",
    "j": "http://release.niem.gov/niem/domains/jxdm/5.1/#",
    "nc": "http://release.niem.gov/niem/niem-core/3.0/#",
    "geo": "http://release.niem.gov/niem/adapters/geospatial/3.0/#",
    "gml": "http://www.opengis.net/gml/3.2#",
    "structures": "http://release.niem.gov/niem/structures/3.0/#",
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  },
  "exch:CrashDriverInfo": {
    "nc:Person": {
      "@id": "P01",
      "nc:PersonBirthDate": {
        "nc:Date": "1893-05-04"
      },
      "nc:PersonName": {
        "nc:personNameCommentText": "copied",
        "nc:PersonGivenName": {
          "nc:sequenceID": 1,
          "rdf:value": "Peter"
        },
        "nc:PersonMiddleName": [
          {
            "nc:sequenceID": 2,
            "rdf:value": "Death"
          },
          {
            "nc:sequenceID": 3,
            "rdf:value": "Bredon"
          }
        ],
        "nc:PersonSurName": "Wimsey"
      },
      "exch:PersonFictionalCharacterIndicator": true
    },
    "j:Crash": {
      "nc:IncidentLocation": {
        "geo:LocationGeospatialPoint": {
          "@context": {
            "srsName": "http://www.opengis.net/gml/3.2#srsName"
          },
          "gml:Point": {
            "@id": "PT01",
            "srsName": "urn:ogc:def:crs:EPSG::4326",
            "gml:pos": "51.835 -0.417"
          }
        }
      },
      "j:CrashVehicle": {
        "j:CrashDriver": {
          "nc:RoleOfPerson": {
            "@id": "P01"
          },
          "j:DriverLicense": {
            "j:DriverLicenseCardIdentification": {
              "nc:IdentificationID": "A1234567"
            },
            "nc:ItemLengthMeasure": {
              "nc:MeasureDecimalValue": 9.7
            },
            "nc:LengthUnitCode": "CMT"
          }
        }
      }
    },
    "j:PersonChargeAssociation": {
      "nc:Person": {
        "@id": "P01"
      },
      "j:Charge": {
        "@id": "CH01"
      },
      "j:JuvenileAsAdultIndicator": true
    },
    "j:Charge": {
      "@id": "CH01",
      "j:JusticeMetadata": {
        "@id": "MD01",
        "j:CriminalInformationIndicator": true
      },
      "j:ChargeDescriptionText": "Wild Driving",
      "j:ChargeFelonyIndicator": false
    }
  }
}

RDF

The NIEM conceptual model is based on the RDF data model described in RDF-Concepts. NIEM defines a mapping from the XML components in IEPs and IEPD schemas to equivalent RDF triples. As of this writing, there is no automated translator for that mapping.

Another reason for choosing JSON-LD is that it is a concrete RDF syntax as described in RDF-Concepts. There is a mapping between JSON-LD objects and equivalent RDF tuples. This mapping is implemented in open-source translators. This means that the JSON-LD serialization of an IEP can be automatically translated to Turtle or RDF/XML and processed in that form.

This establishes two paths from a NIEM IEP to RDF

Direct, using the mapping in the NDR
Indirect, first following the guidance in this document to produce a JSON-LD serialization, then converting that JSON-LD to RDF

The NTAC intent is that the RDF created by both paths will be consistent. This may entail future revisions to the NDR.

Advanced NIEM Concepts Reference Guide Next section