Bussol: PRO Data Parser and Import: Websites, REST API, Excel, JSON, XML, Products, Prices and Stock

What this solution does

Bussol: Data Parser and Import PRO: Websites, REST API, Excel, JSON, XML, Products, Prices, and Stock

A universal module for importing, parsing, and synchronizing data within the 1C-Bitrix environment. The solution extracts information from websites, via REST API, from supplier files, and external systems, transforms it according to defined rules, and automatically loads it into InfoBlocks, the trade catalog, and Highload blocks of the platform.

The tool is optimal for online stores, content projects, catalogs, integration tasks with suppliers, as well as for agencies and developers seeking to replace a set of scattered scripts for HTML parsing, Excel/XML/CSV import, and API synchronization with a single unified solution.

Automation capabilities:

Populating the site with products, news, articles, and other content.

Regular updates of prices and warehouse stock levels.

Import of products and trade offers (SKU).

Synchronization of multiple price types and currencies.

Uploading images, files, and product characteristics.

Creation and updating of InfoBlock sections and properties.

Importing data into Highload blocks.

Retrieving data from REST API with authorization support.

Processing large price lists and catalogs on a schedule.

Monitoring product availability and automatic deactivation of out-of-stock items.

The Bussol: Data Parser and Import PRO module provides mass website page traversal with pagination support and content extraction from resources where data is generated dynamically via JavaScript.

Supported data sources:

HTML websites: usage of CSS selectors and XPath, link traversal, pagination handling, image extraction, and Headless parsing with JavaScript execution.

REST API and JSON: support for GET, POST, PUT, PATCH, DELETE methods, header configuration, Bearer, Basic Auth, and OAuth2 authentication, as well as mapping of nested data structures.

XML and YML: XPath processing, streaming handling of large files, and import of products and offers.

CSV: operation based on column headers, processing of large files, and streaming import.

XLSX and ODS: table import with the ability to select sheets and map columns.

FTP and SFTP: direct retrieval of supplier files from remote servers.

The Visual Selector PRO tool is designed for working with HTML sources. It allows configuring the parser without manually searching for selectors in DevTools. Features include visual selection of page elements, automatic construction of CSS selectors and XPath generation, creation of multiple robust selector variants, counting match occurrences, automatic detection of repeated product cards and list items, and DOM manipulation after JavaScript rendering in Headless mode.

If a site is built on React, Vue, SPA, or loads data dynamically after the page opens, Headless mode is available. The module supports local Playwright and Chromium engines as well as external headless renderers. Standard HTML profiles continue to work without requiring Node.js.

Flexible Data Mapping

Each profile contains rules for mapping source data to 1C-Bitrix fields. Data can be transferred to NAME, CODE, PREVIEW_TEXT, DETAIL_TEXT, as well as PREVIEW_PICTURE and DETAIL_PICTURE. The system supports populating any Information Block properties, including multiple and file-type properties. Synchronization is available for prices and currencies, stock levels and warehouses, catalog trade parameters, SKU and product offer characteristics, and UF fields of Highload blocks.

Data Transformation

Before saving, values pass through a transformation chain, allowing supplier data to be adapted to the site structure without writing separate PHP scripts. Available operations include trim, text replacement, regular expressions, number extraction, rounding, multiplication and division, prefix and suffix handling, date conversion, length limiting, and HTML and text processing.

Price processing example:

Source value: "12 990 rub."

1. Convert to number (number)

2. Multiply by 1.15 (multiply 1.15)

3. Round (round)

Result: 14 939

The module is designed not only for content information blocks but also for full-scale management of a trading catalog. It supports creating and updating products, generating SKU trade offers, and automatically linking SKUs to parent products. The system handles multiple price types with individual currencies for each type, currency conversion, total stock levels, and stock quantities per warehouse. It includes features for managing weight, dimensions, VAT rates, and measurement units, as well as uploading images and files.

The automatic structure creation functionality includes generating sections based on source data with support for nested section trees, creating missing properties and values for list-type properties, and automatically generating simple user-defined fields for Highload blocks.

Before actual synchronization, a Dry-run and Diff mode is available. In this mode, the module analyzes the source data without modifying the site. The analysis results display which elements will be created or updated, which data remains unchanged, and a comparison of old and new field values, including price and currency changes. This feature is particularly useful when launching a new supplier for the first time or after adjusting mapping rules.

The module also handles the logic for removing products that have disappeared from the source.

After a successful import, the module identifies items that were previously present in the source but are no longer there. Available processing modes include: do nothing, deactivate the item, zero out the stock, or deactivate the item and zero out the stock simultaneously. Protective logic prevents mass deactivation in case of source errors, incomplete data traversal, or when import limits are reached.

Automation and Scheduling

Supported launch methods include manual profile execution, Bitrix Agents, cron, custom interval triggering, batch processing, limits on the number of pages and items, retry of failed tasks, and result notifications.

Multi-process Workers

For large catalogs, parallel queue processing is supported using multiple worker processes. Features include configurable worker count, atomic task locking, lease locks, heartbeat monitoring for active workers, protection against duplicate task processing, and automatic re-queuing of stuck tasks.

Launch example:

php /local/modules/bussol.contentparser/tools/cron.php --workers=4

The system provides automatic recovery after failures.

The Watchdog module monitors the status of the task queue and worker processes. If a PHP process, cron job, or server terminates abnormally, the module can return stuck tasks to the queue and resume processing. It also handles orphaned runs where all jobs are completed but the process failed to finish the Run step correctly.

Supported authorization and network settings include Basic Auth, Bearer Token, OAuth2 Client Credentials, arbitrary HTTP headers, and cookies. Connections can be configured via HTTP/HTTPS proxies with authentication, as well as through FTP and SFTP with fingerprint verification. Customizable timeouts are available.

For large files in CSV, XML, YML, XLSX, and ODS formats, the module uses streaming processing to avoid loading the entire file into memory. This enables the use of the module for large catalogs and price lists where standard import via a single PHP process might hit the memory_limit restriction.

Highload blocks can be selected as an independent target within a profile. The Mapping module writes data directly to UF fields. Supported operations include creating new records, updating existing ones, configuring an identification field, and automatically creating simple UF fields.

An independent profile with individual settings is created for each data source, including source type, field mapping, target InfoBlock or Highload block, section, schedule, authorization, proxy, transformation rules, and update policies. Profiles support export and import in JSON format. Sensitive data is removed by default during export.

Administrators receive a unified control interface. The import status dashboard displays recent runs, counts of created and updated elements, the current task queue, and detailed error reports for specific URLs and records. Information on worker statuses is available, along with checks for PHP extensions, iblock, catalog, highloadblock, and currency modules, as well as agent and queue status. The system also performs diagnostics for dependencies, including Headless, FTP/SFTP, and other services.

Upon completion of synchronization, the module can send notifications via Email, 1C-Bitrix administrative notifications, or to an external Webhook. Settings allow for notifications on errors only, successful runs only, or every profile completion.

The module is designed for working with external sources and has undergone a dedicated security audit. It includes protection against SSRF attacks and attempts to access private or internal networks.

The module provides comprehensive data protection and import process security. It implements CSRF checks for administrative actions, access right verification, and protection of sensitive profile data. The system prevents injection attacks in HTTP headers and cookies, validates SFTP fingerprints, and filters dangerous HTML code. Uploaded file types are restricted, and the transfer of authorization credentials to third-party origins is blocked. Secure processing of XML and external files is ensured.

The module is designed for online stores requiring automatic synchronization of products, SKUs, prices, stock levels, and warehouses. Suppliers and distributors can utilize REST API, Excel, XML/YML, CSV, and FTP/SFTP protocols. Content projects can use the tool for automatic loading of news, articles, and catalogs. 1C-Bitrix integrators receive a single tool for handling various data sources instead of writing individual import scripts. Digital agencies can transfer profiles between projects and use them as a foundation for standard integrations.

Key advantages include replacing multiple tools with a single module featuring a unified interface for websites, APIs, Excel, and supplier files. Integration configuration is performed via profiles without rewriting code, where mapping and data transformations are defined. The solution supports real e-commerce scenarios, including handling SKUs, multiple prices, currencies, warehouses, and automatic removal of discontinued products.

The Bussol: Data Parser and Import PRO module is designed for working with modern web resources, including sites with dynamic JavaScript content. It supports the use of headless browsers and the advanced Visual Selector PRO. The architecture ensures scalability through task queues, streaming data processing, and multi-process workers.

The system includes pre-execution control mechanisms: Dry-run mode and diff comparison allow administrators to review changes before applying them. Reliability is ensured through automatic retry logic, task leasing, heartbeat monitoring, protection against hanging processes, and recovery of interrupted tasks. The administrative interface provides a dashboard, detailed logs, diagnostic tools, and a notification system.

A typical import scenario involves querying a supplier's REST API, authenticating via OAuth2, and receiving data in JSON format. Mapping rules and transformations are then applied. Before writing, a Dry-run and diff check is performed. The system imports products with SKU identifiers, retail and wholesale prices, and warehouse stock levels. Data is processed through a worker queue and automatically synchronized with the 1C-Bitrix platform.

Environment requirements:

Basic functionality operates in a standard 1C-Bitrix configuration. Extended features require the following components:

- DOM, XMLReader, and ZIP libraries for handling XML, XLSX, and ODS formats;

- The Trade Catalog module for managing products, prices, and SKUs;

- Highload blocks for assigning high-load tasks;

- The Currencies module for currency conversion;

- A Node.js environment with Playwright or Chromium libraries, or an external renderer, for local headless JavaScript parsing;

- PHP FTP and SSH2 extensions for file transfer protocols.

Bussol: Data Parser and Import PRO is a universal platform for automating the retrieval, transformation, and synchronization of external data with 1C-Bitrix. The tool supports scenarios ranging from simple text article imports to regular synchronization of large product catalogs. Supported data sources include OAuth2 REST API, Excel files, XML and YML formats, and the SFTP protocol. All operations are configured through unified profiles, field mapping, task queues, and a scheduling system. After the initial source configuration, the module automatically maintains data currency on the website.

General iT can connect “Bussol: PRO Data Parser and Import: Websites, REST API, Excel, JSON, XML, Products, Prices and Stock”, install and configure the module, and make sure the integration works correctly.