Deep Dive: JSON Stored Content-Items (Entities)

In 2sxc 9.4 we introduced a new format for storing entities as JSON. In the future, they will become extremely important, so we'll give you a deep-dive into how it works. 

JSON Entities Everywhere

JSON entities are used in many places, for example:

1. in the history - where we keep old copies of entities
2. in the DB, when storing items with dynamic (unknown) types
3. in the DB, when storing items with types which are known, but not registered in the DB
4. in the file system
5. in various export/import scenarios - like when exporting a query

JSON Entity Format

Here's what an entity can look like - in this case it's an entity describing a visual query:

{
    "_": {
        "V": 1
    },
    "Entity": {
        "Id": 42900,
        "Version": 6,
        "Guid": "e8a702d2-eccd-4b0f-83bd-600d8a8449d9",
        "Type": {
            "Name": "DataPipeline",
            "Id": "DataPipeline"
        },
        "Attributes": {
            "String": {
                "Description": {
                    "*": "Retrieve full list of all zones"
                },
                "Name": {
                    "*": "Zones"
                },
                "StreamsOut": {
                    "*": "ListContent,Default"
                },
                "StreamWiring": {
                    "*": "3cef3168-5fe8-4417-8ee0-c47642181a1e:Default>Out:Default"
                },
                "TestParameters": {
                    "*": "[Module:ModuleID]=6936"
                }
            },
            "Boolean": {
                "AllowEdit": {
                    "*": true
                }
            }
        },
        "Owner": "dnn:userid=1",
        "Metadata": [

        ]
    }
}

The format looks like this:

• _ (header) mainly storing the version, in case we have to introduce a breaking change
• Entity - this marks an entity - at the moment a json package should only have 1, but later it could contain more
  • Id - the identity as a number
  • Guid - the identity as a guid
  • Type - type information
    • Name - the type name
    • Id - the type identity - usually a guid, but special types can also use a specific string
  • Attributes - the values of this entity
    • String - all the string values
      • [the field name]
        
        • [the languages this value applies to]
          • [the value]
        • [more languages / values]
      • [more fields / languages / values]
    • Boolean - all the boolean values
    • Number - ...
    • [more types]
  • Owner a special string identifying the owner of this item
  • Metadata (optional, array of more entities) - a list of items which further describe this entity
    • [item 1]
      • Id
      • Guid
      • ...
    • [next items]

The Special Stuff about the JSON Format

Here are some things that may seem unusual at first:

All Attributes are Grouped by Type

Because JSON is itself a very loose data-format, and certain types like dates are not auto-detectable, we decided to have the type-specification as a first-class citizen in the format. This allows for automatic, reliable type-checking when materializing objects. 

All values have language information

As we're usually working with real-life content-items, multi-language is always a concern. Because of this, every value is multi-language by default. If the language code is *, that means that this value is the default/fallback value for all languages. 

Metadata is a Recursive List of Entities

2sxc and the EAV is all about real-life content-management. As such, many pieces of information have more information attached, called Metadata. Metadata-items could themselves have their own Metadata, which is then of course attached as well. 

TL;DR

With coding-love from Switzerland,

Daniel