PubMag is growing: 4,000 domains, news, new filters, and performance boosts

PubMag is growing: 4,000 domains, news, new filters, and performance boosts

Over the past month, several updates have taken place that I’d like to share. The database has expanded, interfaces have improved, processes have accelerated — and, inevitably, new bugs have emerged.

📈 Database and Collection Dynamics

  • The database of header bidding auctions has surpassed 4,000 unique domains. Thanks to the pirates!
  • Over 2,000 search queries have been collected, of which fewer than half have been processed and tagged for whitelists — a large-scale task lies ahead, although the process is already largely automated.
Contextual whitelists

🧩 Improvements to the Whitelist Constructor

The filter interface now includes a check by country using .RU domains. Planned additions:

  • A button to load more domains based on current filters
  • Option to hide service websites (Telegram, Wikipedia, etc.)
  • A filter for sites with confirmed advertising activity. Activity will be determined not just by the presence of header-bidding.js, which can be obfuscated, but by a combination of multiple signals. The monetization profile on the domain pages will also be updated. Incidentally, the goal of this profile is to communicate with visitors in human language, not dry reports. This approach might become the default.

🕵️ Update Dates and Transparency

The domain page now shows the date of the most recent report, allowing users to assess the relevance of the information.

Marking Bureau: putting search queries to work

🧠 LibTracker: Libraries, Syncs, and Speed Boost

  • More than 100 new libraries (3x more than before), cookie syncs, and auxiliary solutions have been added to LibTracker.
  • The crawling engine has been optimized — it’s now faster and more resource-efficient.

Top 10 sites by number of detected technologies:

  1. elc-russia.ru — 39
  2. paparazzi.ru — 37
  3. tennis-score.pro — 36
  4. starhit.ru — 37
  5. calend.ru — 35
  6. prigotoovim.ru — 34
  7. maximonline.ru — 34
  8. ngs24.ru — 34
  9. ormatek-com — 34
  10. russian7.ru — 33

Going forward, the plan is to better differentiate real libraries, cookie syncs, and other auxiliary requests. Otherwise, the picture can vary even on the same website — for example, “Detsky Mir”: elc-russia.ru and detmir.ru. The reports don’t match, although it’s the same site.

📰 News and Telegram Bot

The short news section received the following updates:

  • Telegram bot launched for quick news submissions
  • Main news page redesigned
  • Automatic tagging configured with minimal manual correction

After several iterations of development, stability and content expansion have become top priorities for the section.

🔐 Authorization: Gmail Issues

Magic Link login remains unstable for Gmail users — emails often don’t get delivered. No solution has been found yet.

🏠 Server and Automation

  • A remote management module has been configured on the home server and a working Telegram bot has been launched for the AstraLab team.
  • LibTracker is being prepared for migration to the home server, following the example of HBTracker, to automate data collection.
Autonomous operation of HBTracker

📊 Charts and Technology Pages

The chart module redesign is nearing completion. Once the database is optimized, it will be possible to display full lists of sites for each technology — not just the last 50. Similar changes are planned for LibTracker data as well.

This sums up the end of April and beginning of May. Come say hi — I really need your feedback! Wishing everyone both money and knowledge!

January Results and February Plans January Holidays How I linked data from HBTracker and LibTracker and what came out of it Autumn Update of Advertising Analysis Tools