Overview 

As part of our ongoing work to provide greater clarity around system limits we are defining the limits when importing data. By defining these limits, we will be able to provide improved error handling and feedback. 

Who will this affect?

This change will affect anyone using the Alloy imports to perform large scale imports. Note that the limits are defined with some headroom such that imports with records counts over the set limits that may have previously passed will now fail.  

Details

Data imports with no parent links configured will be limited to 1,000,000 records.

Data imports with any parent link configured will be limited to 500,000 records. This is due to the extra processing and storage overhead required to connect items as part of an import.

To import more than the above specified items in the system, it will be necessary to stage the process and execute multiple imports.

The parent links mentioned are ones that can be configured in the "Parents" section within the Gateway module:

Or for API users, by using the parents property in the ImportValidateWebRequestModel:

{
  "designCode": "designs_test",
  "collection": "Live",
  "mode": "Insert",
  "settings": {
    "attributes": [],
    "networkReferences": [],
    "parents": [
      {
        "dodiCode": "designs_parent",
        "attributeCode": "attributes_link",
        "matchHeader": "MyHeader",
        "matchAttributeCode": "attributes_random"
      }
    ]
    "discriminator": "ImportSettingsDataWebModel"
  },
  "signature": "60e623e00000000000000000"
}

Expected Release Date

30th September 2021