pyHQL (ASAM ODS Toolbox for Python) enables analysts and engineers to utilize the HighQSoft Query Language (HQL) to work directly with ASAM ODS data from Python, providing typed results, mass-data access with units, transactions, and file streaming for analysis, scripts, and dashboards.

HighQSoft is introducing pyHQL, a lightweight Python library that connects to the HQL WEB Service and, through it, to any ASAM ODS server in your infrastructure. You use familiar HQL commands to query metadata and measurement data. pyHQL maps ASAM ODS datatypes to convenient Python objects, allowing you to analyze them immediately.

pyHQL eliminates the effort of crafting low-level REST calls and interpreting ASAM ODS datatypes by providing a ready-to-use library—so teams can spend time analyzing data and focusing on insights rather than integration work.

For End-users

• Query information easily — Use the same HQL queries you already know to retrieve any information from ASAM ODS servers - with all the comfort you'd expect!

• Work natively in Python & Data Frames — Stay in your comfort zone with Python, Pandas, and friends.

• Transactions & file streaming — Open/commit/abort transactions from Python; upload and download files via dedicated helpers. Uncommitted transactions are safely aborted on disconnect.

For Analysts & Data Scientists

• Create dashboards — Build dashboards on ASAM ODS data with Plotly or other tools on top of typed results.
• Write helper scripts — Quickly retrieve and prepare ASAM ODS data for engineering teams or process information for other stakeholders.
• Automate pipelines — Combine pyHQL capabilities with Merlin’s pre-built stages to retrieve and write information from ASAM ODS and implement automated processing pipelines.

For Administrators

• Product Stability — Built on the well-maintained HQL WEB Service for consistent, predictable behavior.
• Simplicity — Get users started fast with a straightforward, two-URI connection model.
• Compatibility — Works with ASAM ODS 5 and ASAM ODS 6.

pyHQL is available now as a ZIP for pip install from the HighQSoft download page. The manual includes installation steps, connection examples, and a quick tour of queries, mass data access, transactions, and file transfer.

#ASAM #ODS #TestData #Merlin #Python #pyHQL #TDM #TestDataManagement #ASAMODS #HighQSoft

HighQSoft is introducing the Merlin UI, a new module for ASAMCommander that brings server-side analysis and job orchestration right into the end-user’s web application. Engineers can discover data, launch analyses, and monitor pipelines end-to-end—without leaving ASAMCommander.

Why it matters

As data volumes grow, manual, desktop-based analysis tools don’t scale. Merlin is a second-generation, technology-independent framework that standardizes execution so teams get faster, comparable, and reproducible results.

What's new?

The following features are introduced:

For End-users

• Start Job toolbar action available across multiple modules to launch analyses from within ASAMCommander.

• Job bookmarks to quickly re‑run jobs or review prior results.

For business admins

• Template Manager to control which job templates are available to end users.

• Template Builder (Code & Graphical) to design and operationalize evaluations.

• Job Monitoring & Logging for live status, history, and traceability.

• Data Lineage to trace software, processes, and all interactions with your datasets.

What you can do with it

ASAMCommander is a modular ASAM ODS web application for data exploration and management. Out of the box, ASAMCommander already integrates expert tools like AVL Concerto, MATLAB, and NI DIAdem, and supports exporting ATFx, CSV, and MDF4—so you can keep preferred workflows in one place.

Merlin extends this with:

• Technology-independent executors (Python, Java, MATLAB, Spark)

• Resource management, queuing/scheduling, and event triggers (e.g., on import)

• Standardized, comparable results with full data lineage

Template types supported include importing, data validation, processing/analysis, statistics generation, archiving, and maintenance tasks.

About Merlin/Sam Analysis Server

Merlin is a second-generation, technology-independent framework for server-side analyses that orchestrates jobs, schedules execution, and scales across runtimes. It brings the algorithm to the data and ensures reproducible results with data lineage. Merlin integrates seamlessly with ASAM ODS-based test data management.

#ASAM #ODS #TestData #Merlin #DataLineage #TDM #TestDataManagement #ASAMODS #HighQSoft

This update brings a host of new features and improvements. Here’s a detailed look at what’s new:

Major Version Update

We have updated our versioning system from 3.x to 25.x, with the main version number now derived from the current year. This change simplifies license management, as a license is valid for the same major and minor version numbers. This update includes various bug fixes to improve stability and performance, optimizations to enhance the overall efficiency and speed of the system, and a consolidated versioning process for better clarity and management.

Single-Sign-On with HQL

HQL now supports both Basic Authentication and OAuth 2.0 authentication strings. While Basic Authentication is the standard for ODS, OAuth 2.0, though not yet defined in the standard, can be used with our ODS server via HQL. This allows for seamless single sign-on capabilities, offering a modern and secure authentication method that keeps up with the latest security standards and simplifies user access.

Flexible ODS Client Development with HQL

This version enhances transaction handling with three new options:

• Transactions controlled by HQL for better consistency.
• Ad-hoc transactions within the current session for flexibility.
• Transactions created externally and used by HQL for integration with external transaction management systems.

These improvements allow for only one transaction per ODS session, enhancing reliability and control.

ASAM ODS Toolbox for Matlab / HQL

The HQL toolbox now includes a statement builder object, which avoids the need to interpret HQL statement strings. This optimization improves data writing efficiency and enhances channel data manipulation. Additionally, MATLAB code can now avoid user code injections, leading to faster and more efficient data processing. Sample code is provided to help users get started quickly.

Matlab Integration in ASAM ODS

We have introduced two new objects for the MATLAB HQL Toolbox that utilize the ASAM ODS submatrix and measurement. These objects load channel data and its metainformation, providing data plotting methods. Key features include:

• Marked independent channels for plotting (x-axis).
• Loaded channel units retrievable from the matrix.
• Channels in a matrix marked by a common background color in plots.
• Descriptions read from the basic attribute of the measured quantity and assigned to the channel.
• Basic statistics such as min, max, stddev, and average are determined and displayed in plots, offering quick insights into the data.

This integration simplifies the data loading process, makes data visualization straightforward and efficient, and is ready-to-use for data science. Full-fledged MATLAB integration in ASAMCommander 2 (toolbar) enhances the user experience, and the capability to save .mat files from ODS provides flexibility in data storage and sharing.

#HQLRelease #SoftwareUpdate #MatlabIntegration #ASAMODS #HighQSoft

23 participants from 9 companies in 4 countries meet to test ASAM ODS client applications, ASAM ODS servers, and ATFX files for compliance with the ASAM ODS standard.

As in previous cross-tests, the participating companies came together to test and improve their software in real-life scenarios. The participants helped each other to understand the small differences in the data formats of the various programs and to solve the difficulties that can arise when interpreting this data. They were also able to discuss ambiguities and contradictions in the standard documentation, which can then be eliminated in the next version of the standard.

Once all the servers in the network had been set up, the first client connections were established. At the end of the first day, all clients were able to connect to all servers.
Each participating client was assigned an Excel sheet with the scenarios to be tested for each server. This time, the participants agreed to test only the http interface of the ODS standard. The scenarios were, for example:

• Opening the session
• Reading context variables
• Browsing through the test hierarchy
• Reading the model including the n to m relations
• Simple and nested queries
• Reading attributes and mass data including size and unit references
• Writing new instances and mass data
• Handling of external files (AoFile)

The tests were then carried out on the second day. Groups were formed with at least one representative each from the client manufacturer and the server manufacturer. A few identified problems were posted directly as issues in the ASAM GitHub of the ASAM ODS working group to be resolved with the upcoming release.

Today, we released a number of improvements for our products

ASAMCommander: Catalog Editor

The release includes several enhancements for managing catalogs in ASAM ODS (Physical Dimensions, Units, Quantities, and groups). The  Catalog Editor provides comprehensive CRUD (Create, Read, Update, Delete) operations and additional features like transaction control, drag and drop functionality, and support for managing catalog elements and their instances.

HQL: Transaction Handling for Outer Sessions

This update allows creating transactions outside of HQL and requesting HQL to use them. This lifts the limitation of only one transaction per ASAM ODS session.

Merlin/SAM: New Validation Job Pipeline

A new Validation Job Pipeline runs custom Python scripts on imported data if certain criteria are met. The scripts can generate new parameters or measurements based on the data.

Merlin/SAM: Check New Job Pipeline for Filter Criteria

Introduces a Job Pipeline for checking newly imported measurements against filter criteria stored in the ASAM ODS database. The pipeline stages include applying filter criteria, copying processing results, and triggering validation.

Merlin/SAM : Adaptation to Python Repository

Implements a git storage for loading remote stage functions for Job Pipelines making it easier to manage and update scripts.

All updates enhance the functionality and flexibility of the Test Data Management Ecosystem providing robust tools for data validation, transaction handling, and catalog management.

#HighQSoft #Release #HQL #SoftwareUpdate #ASAMCommander #ASAMODS

What’s the Deal with ASAMCommander? 

Test Data Management can be a headache. Multiple proprietary formats, incompatible data structures, and a never-ending stream of new information can make it difficult, if not impossible, to turn raw data into actionable knowledge.
At HighQSoft, we specialize in wrangling your measurement data into something you can use today and in the future. We offer multiple products and solutions to help you organize and manage data, regardless of volume and format.

Our ASAMCommander is a test data management application based on the ASAM ODS standard (https://www.asam.net/standards/detail/ods/). As a modular web application, it comprises loosely coupled, functional units called modules that form a customizable experience for end-users, system and data administrators, or data analysts.

The basic version of ASAMCommander is an out-of-the-box application that provides all end-user use cases for data exploration (navigation, search, visualization), expert tool integration (e.g., AVL Concerto, Mathworks Matlab, NI DIAdem), and data exports (ASAM ATFx, CSV, ASAM MDF).

The Enterprise Edition of ASAMCommander is suited for Global Test Data Management solutions based on the AReS Libertas ODS6 Platform. System administration features include system logging, data lineage, and data retention. Our Merlin Analysis Framework provides all features for complete automation of analysis pipelines. ASAMCommander Enterprise allows customization, and new modules, features, and workflows are easy to add or integrate.

ASAMCommander 24.2: Enhanced Features and Improved User Experience

The ASAMCommander release 24.2 is now available at our download portal and includes:

AC-Mainframe

• OAuth Authentication
  Support for OAuth authentication.
• Icon Size Fix
  Ensured consistent icon sizes across all (module) cards.
• Bug Fix
  An issue with highlighting and removing background color after clicking on a card and dropdown options was resolved.

AC-Navigator

• Tree Search Fix
  The tree generated after ‘Search’ now closes appropriately, and the ‘load more’ button is only visible when more children are available.
• Attributes Request
  Added the functionality to request attributes for application elements and display them in the Node Overview.
• Copy to Clipboard
  Implemented a feature to copy attributes directly to the clipboard.
• ASAM ODS Attributes
  Back-end support was added to show ASAM ODS attributes when selecting an ASAM ODS element.

AC-Logger

• Increased Usability
  Developed UI to indicate the current page after searching or sorting in the logs table.

General Features

• Extended CSV Exporter
  Integrated the support to export multiple measurements when exporting from a higher logical level.

The release contains various enhancements and updates for AC-MoMa, AC-Bookmarks, AC-Search, and AC-Merlin.

ASAMCommander works with any data model (ISO Safety, ASAM NVH, custom models), all domains (battery, powertrain, vehicle, ...), and any ASAM ODS server (AReS ODS Server, AVL Santorin).

#HighQSoft #TestDataManagement #MeasuredDataManagement #ASAMODS #ASAM #Release #Innovation

Libertas 24.1.3 brings a series of significant improvements and new features designed to enhance your experience. Here are the highlights of this release:

• Bearer Token Authentication: In addition to Basic Authentication, users can now log in using Bearer tokens (JWT), providing more flexibility and security in accessing the system.
• String Length Compliance: Libertas now strictly enforces string length limits, aligning with industry standards to ensure data validity.
• New String Data Support: This release includes support for additional string data types, enhancing the system's capability to handle diverse data formats.
• UTF-8 Character Set Support: Full support for the UTF-8 character set is now available, ensuring proper handling and display of international characters.
• Automatic User Group Assignment: New users authenticated via OAuth can now be automatically assigned to a default user group, simplifying user management and access control.
• Consistent File Downloads: Downloaded files will now consistently use the application/octet-stream content type, following industry recommendations for better compatibility.

These enhancements and fixes are part of our ongoing commitment to improving Libertas, making it more robust, secure, and user-friendly.

#HighQSoft #SoftwareUpdate #Libertas #Innovation

We are pleased to announce the release of ASAM Commander 24.1, packed with enhancements geared towards boosting stability and enhancing user experience across various modules, including some design upgrades throughout the entire application.

Web Application Enhancements

• Commander: Now supports localization in both German and English, alongside fixes for reload/refresh functionality, and the addition of module highlighting in the Navbar for better navigation.
• Navigator: Has undergone improvements, addressing issues with the Export feature within the Navigator Tree, refining the instance editor, and rectifying data type inconsistencies.
• MoMa: Now includes Elapsed Time in its monitoring table, enhancing its functionality.
• Bookmarks: We've upgraded the Bookmarks feature with keyword highlighting, improving search clarity.
• Search: Fixed pagination issues within Advanced Search, ensuring smoother navigation through search results.
• Merlin: We've resolved issues related to the Run feature in the HQL Job Type, ensuring seamless operation.
• Logger: Say goodbye to hostname identification issues! We've enabled container identification by name, ensuring accurate logging and traceability.

#HighQSoft #SoftwareUpdate #DataManagement #ASAMCommander #Innovation

Libertas 23.2.2 introduces several key fixes and improvements aimed at enhancing the performance and stability of the platform. Here's a summary of the notable changes:

• Performance Improvements: The data-create endpoint for PostgreSQL now benefits from multi-value insert statements, resulting in significant speed enhancements, with observed import time reductions of up to 50% in test environments. Additionally, batch size configurations for both data-create and data-update endpoints have been introduced, providing flexibility and optimization options.
• Model Read and Data Update Fixes: The release resolves problems with model reading for non-super users and ensures correct behavior of the data-update endpoint when handling empty values.
• Database Connection Pooling: A production-ready database connection pool has been implemented to increase connection reuse and mitigate the risk of reaching maximum connection limits. Configuration options are available via Manatee settings.
• OSGi Shell for AReS: A new OSGi shell for AReS has been introduced under the 'ares' command, enabling management of Libertas connections and resources via CLI.
• Database Recovery and Session Timeout Configuration: Libertas runtimes now support automatic recovery after database restarts, minimizing service interruptions. Session timeout mechanisms can also be configured via 'config.xml' files.
• Enhanced MS SQL Server Support: Improved support for MS SQL Server includes better transaction and connection handling, handling of large parameter queries, and ensuring compatibility with MS SQL Server SQL dialect.
• Tagging Support: Libertas 23.2.2 introduces support for tagging via HTTP headers, facilitating integration with tools like Postman.

#Libertas #LibertasRelease #SoftwareUpdate #HighQSoft

We're thrilled to announce the latest release of ASAMCommander, packed with numerous enhancements and improvements that promise to elevate your user experience to new heights. Our team has been hard at work in the AC 23.3 work package, focusing on stability, performance, adding new functionalities and refining overall design to ensure you get the most out of ASAMCommander.

Web Application Enhancements

• Navigator: Experience enhanced data visibility across various tabs, including Data View, Attributes, Instances Navigation, and Attachments. Effortlessly export measurement data directly from the tree nodes.
• MoMa: Benefit of the addition of importer information, capability to import big files, job duration visibility, and enhanced monitoring capabilities.
• Bookmarks: Bookmark the same element in different lists, with fixes and enhancements for smoother navigation and export capabilities.
• Search: Navigate through an advanced search with a new design and fixes for better search experience.
• Merlin: Explore improvements including fixes for evaluation creation, addition of HQL job type, and enhanced upload features.
• Logger: Gain insights with container name additions to logs table and improved filtering for better analysis.

These enhancements are geared towards making your experience seamless and efficient, empowering you to manage your data effectively.

Happy exploring!

#HighQSoft #SoftwareUpdate #DataManagement #ASAMCommander #Innovation