Within the early days of ASAM ODS, Test Data Management systems started as single data repositories for independent projects. Over time, projects were consolidated, for example, the engine and transmission domains into the powertrain project. Today, enterprise system consolidates the data of corporate, various sources including vehicles, multiple user groups and stakeholders, and automated analysis frameworks.
Our Avalon ODS Server always featured a parallel instantiation to e.g. channel workload from import processes, users, and analysis. In 2015, we introduced the Avalon Distributor as a single gateway for ASAM ODS 5 access. The key feature of the Avalon Distributor was the load-balancing of queries utilizing multiple Avalon ODS Servers. Moreover, you can use the distributors with fallback options, so the whole server infrastructure is kept alive. Thus, ASAM ODS offers horizontal and vertical scalability for test data management early on. The feature was continuously further developed and is still an integrated component today.
A remaining limiting factor to ASAM ODS 5 systems is the CORBA dependency prohibiting newer containerization technologies such as Docker and Kubernetes.
In 2019/2020, the ASAM ODS 6 API was introduced. This development is a gamechanger as it provides a modern HTTP REST API to enable the integration of test data management systems into today's infrastructures. Applications based on web platforms, such as our Manatee Web Application, provide data access from anywhere in the world. Additionally, and next to the ASAM ODS 6 Web Service, the HighQSoft Query Language Libraries (HQL) enabled developers and engineers to integrate their tools, platforms, or analysis into ODS without greater efforts or ODS API know-how - making data more accessible than ever.
A rather recent aspect of ASAM ODS is the integration of automated evaluations. While the engineer may still use the preferred tool to develop algorithms, frameworks like Merlin integrated the algorithms into a server si process and bringing the algorithm to the data. Incoming data now gets automatically validated and prepared for standardized reports. This was an important step towards performance, comparability of results, and avoiding data redundancy.
All those aspects of a solution integration match the state of the art in ODS and have enough performance for most cases within the ASAM ODS domain.
Initial Participants
Audi
AVL
BMW
Bosch
Cummins
ETAS
Ford Motor Company
GM
HighQSoft
IASYS
Müller BBM
National Instruments
Peak Solution
PSA
RD Electronic
Vector
White Pine
To fulfill the requirements of large-scale data management and big data analysis, the domain and all its members formed an initiative to further develop the ASAM ODS standard. Arranged by Cummins in the USA, the initiative had members like AUDI, Cummins, ETAS, Evolutionary Software, Ford, General Motors, HighQSoft, RDE data solutions, White Pine Software Technologies, and others.
As a result, the ASAM ODS working group defined three formats that integrate well with the big data world: A JSON format to integrate metadata into indexer services, an AVRO format for data transportation, and a PARQUET format for data storage in Hadoop.
ASAM ODS 6 comes with an HTTP API that allows integration or a combination of none-SQL data repositories. To interconnect with today’s world of web services, the ASAM ODS working group defined a JSON schema to export data into other repositories or indexer technologies. The main purpose of the JSON format is the interchangeability of meta-data.
With the Parquet file format definitions, the ASAM ODS Working group defined a format that can easily be integrated within the Hadoop Ecosystem. It contains the mass data/channel data that can be accessed with the information stored in the JSON, or yet again ATFx files. This opens up the possibility to access data with other interfaces or query languages preferred by data scientists. At the same time, the format definitions guarantee that data always is transparently stored.
With the AVRO file format definitions, the ASAM Working group defined an alternative format for Parquet. The Parquet format can be used for data storage but primarily fulfills the use-case of quickly writing and transporting data when it is generated. The AVRO format can be transformed to the Parquet format and vice versa. It works in combination with the same JSON definitions. Both are single node entry points to scale the test data management system with increasing requirements. Also, an IIOP Gateway is available to provide a migration scenario for ASAM ODS 5 to ASAM ODS 6 systems. The IIOP Gateway provides the same features as the other gateways.
HighQSoft GmbH
Black-und-Decker-Straße 17c
D-65510 Idstein
