Introduction

This page explains how the output-module validates your documents before they are sent to Shopware. The output-module validates each document to check if it can be transformed into a valid Shopware entity. Examples for validation are:

  • If strings are too long for the target field
  • If strings contain html-tags which is not allowed by Shopware
  • If values are logically correct, e.g. if a product stock or dimension is not negative
  • If translations contain values for the system language which is required by shopware
  • If identifiers are unique
  • If referenced Shopware entities (for example the "parent" property option of a property value) are available in Shopware
  • ...

The validation takes place in the -validate subsections (e.g. product-validate).

Upsert-Progress Manipulation

If a document does not pass the validation, the upsert-progress of the affected subsection is reduced. So at the end of a -validate subsection the -upsert subsection is reduced by the count of invalid documents.

Example: If the product-validate finds 10 invalid products the total progress of the product-upsert subsection is reduced by 10.

Logging Validation Infos

The validation itself can not be deactivated. However, you can configure in which way the generated validation messages are logged and/or persisted. The following options are available:

  • write validation infos to console and log files
  • persist validation infos in your documents to make them visible in the frontend
  • generate validation files (.txt/.csv) that contain all validation infos

This can be managed by the following configuration that will be explained in detail in the following sections:

{
  "validation": {
    "logging": {
      "validDocuments": false,
      "invalidDocuments": false,
      "severityLevels": [
        "info",
        "warning",
        "alert"
      ]
    },
    "persist": {
      "validDocuments": false,
      "invalidDocuments": false,
      "severityLevels": [
        "info",
        "warning",
        "alert"
      ]
    },
    "fileGeneration": {
      "generateFiles": false,
      "fileTypes": [
        "csv",
        "txt"
      ],
      "severityLevels": [
        "info",
        "warning",
        "alert"
      ]
    }
  }
}

Logging

The logging key of the configuration manages if/how validation reports are printed to console and debug logs:

{
  "validation": {
    "logging": {
      "validDocuments": false,
      "invalidDocuments": false,
      "severityLevels": [
        "info",
        "warning",
        "alert"
      ]
    }
  }
}
  • validDocuments (de)activates the logging of reports for valid documents (even valid documents can contain warnings, e.g. for missing system translations)
  • invalidDocuments (de)activates the logging of reports for invalid documents
  • severityLevels determines the severity levels to include in the reports

The module generates a validation report for every document. This report is logged and could, for example, look like this:

13:39:15 WARNING   [ValidationModuleLoggingService] Product (id: 6017ce8df24ea17750203293, identifier: 123456) is invalid and not mappable: {  
    "alerts": [  
        "source 'taxGroup' mapped to 'ProductEntity::taxId': Required value is not available (null or empty)",  
        "source 'availability.warehouses' mapped to 'ProductEntity::stock': Required value is not available (null or empty)",
        "source 'packageDimensions.width.value' mapped to 'ProductEntity::width': The target field does not allow negative numbers",
    ],  
    "warnings": [  
	    "source 'generalInformation.description' mapped to 'ProductEntity::description': This collection does not contain a value for the default language (de-DE)"  
	    "source 'generalInformation.name.de_DE.value' mapped to 'ProductEntity::name': This string is not trimmed"
	],
    "infos": [  
        "source 'packageDimensions.width' mapped to 'ProductEntity::width': Optional value is not available (null or empty)"
    ]  
}

Reports for invalid documents are logged as WARNING, reports for valid documents as INFO.

Persist

You can also let the module persist validation infos to make them visible in the frontend. The configuration looks like this:

{
  "validation": {
    "persist": {
      "validDocuments": false,
      "invalidDocuments": false,
      "severityLevels": [
        "info",
        "warning",
        "alert"
      ]
    }
  }
}
  • validDocuments (de)activates the persisting of reports for valid documents
  • invalidDocuments (de)activates the persisting of reports for invalid documents
  • severityLevels determines the severity levels to persist

File Generation

Please keep in mind, that this feature was implemented before the frontend was able to display validation infos. Nevertheless, it stays in the module as auxiliary/debugging function. This feature is no longer being developed.

You can also write the validation reports to .txt and .csv files which gives you a compact overview of the health of your data. However, it is strongly recommended to generate these files only on demand and not permanently. This is to be regarded more as an auxiliary/debugging function with which the status of the transfer database can be investigated manually. For example, this can be helpful for input-module-developers, if they want to check why almost every document is flagged as invalid. This is also the reason why the location of the files is not configurable yet.

There is one file generated for each document type and flow execution. The files are located in vendor/synqup/shopware-6/src/Output/Core/ValidationService/FileOutput.

.txt

The text file is basically a collection of all logged validation messages and has the following structure:

Product (id: 6017ce8df24ea17750203293, identifier: 123456) is invalid and not mappable: {  
    "alerts": [  
        "source 'taxGroup' mapped to 'ProductEntity::taxId': Required value is not available (null or empty)",  
        "source 'availability.warehouses' mapped to 'ProductEntity::stock': Required value is not available (null or empty)",
        "source 'packageDimensions.width.value' mapped to 'ProductEntity::width': The target field does not allow negative numbers",
    ],  
    "warnings": [  
	    "source 'generalInformation.description' mapped to 'ProductEntity::description': This collection does not contain a value for the default language (de-DE)"  
	    "source 'generalInformation.name.de_DE.value' mapped to 'ProductEntity::name': This string is not trimmed"
	],
    "infos": [  
        "source 'packageDimensions.width' mapped to 'ProductEntity::width': Optional value is not available (null or empty)"
    ]  
} 
---------------------------------------------------------------------------------------------------------------------------------------  
Product (ObjectId: 6017ceaaf24ea1775020330a) is valid but contains 5 validation infos: {  
    "warnings": [  
        "..."  
    ],  
    "infos": [  
        "...",  
    ]  
}  
---------------------------------------------------------------------------------------------------------------------------------------
...

.csv

The .csv file contains the validation messages as csv-values and can look like this:

"severity level";"document class name";"document identifier";"source path";"mapped to";"root document class (embedded only)";"root document id (embedded only)";"root document identifier (embedded only)";"validation message"  
info;PropertyValue;375d4e1b470efc9bed5bd765be6cb888;propertyOptionCollection.Farbe.propertyValues.375d4e1b470efc9bed5bd765be6cb888.label.de_DE.value;PropertyGroupOptionEntity::name;Product;601473e4ad1c663f9d59a04b;930596;"This string is not trimmed"  
alert;PropertyValue;f640ae61006e3128d97d835feab6a551;propertyOptionCollection.Farbe.propertyValues.f640ae61006e3128d97d835feab6a551.label.en_GB.value;PropertyGroupOptionEntity::name;Product;60147064ae424f524f2833ce;342086;"Required value is not available (null or empty)"  
alert;PropertyValue;efc5f4b470619b932aed03953e50ce60;propertyOptionCollection.Farbe.propertyValues.efc5f4b470619b932aed03953e50ce60.label.en_GB.value;PropertyGroupOptionEntity::name;Product;601473d44023dc5bb34411ac;836013;"Required value is not available (null or empty)"
On this page