LNDS and DataThings Join Forces to Improve Access to Weather Data in Luxembourg
Weather data is generated continuously across Luxembourg every five minutes, from dozens of public weather stations, year after year. This data is rich and valuable, and represents an important asset for public as well as private services, for research and innovation. As usage needs evolve, ensuring that this data can be accessed in a uniform way and processed efficiently at scale becomes increasingly important.
In this context, the Luxembourg National Data Service (LNDS) partnered with DataThings, a local technology innovator, to explore advanced data processing and infrastructure approaches for meteorological data. By combining LNDS’ role in data governance with DataThings’ expertise in analytics and data technologies, the collaboration focuses on enabling standard, efficient, scalable, and sustainable access to historical weather data.
A growing challenge at scale
The meteorological data ecosystem in Luxembourg is both extensive and complex. Around 50 heterogenous public weather stations operate across the country, each collecting up to 28 different sensor values every five minutes. Over a five-year period, this results in more than 26 million records per measurement set, stored across multiple CSV-based datasets.
In practice, many queries involve analysing large volumes of historical data for a specific place over long periods of time. Supporting such use cases requires data processing approaches that are well suited to time-based data and location-aware data. While traditional relational databases are widely used, large-scale temporal analytics often benefit from alternative architectures that are designed specifically for high-frequency, time series data with a geographic dimension.
Evaluating the possibilities of temporal graph technologies
To address these needs, LNDS and DataThings worked together to set up a proof of concept and to jointly explore what a more unified approach to weather data processing and delivery in Luxembourg could look like. Within this context, GreyCat was selected as the technical foundation for the experiment.
GreyCat is a programmable temporal graph technology developed by DataThings. It enables analytics to be executed directly alongside the data, reducing the need for large data transfers and allowing computations to be performed more efficiently. By scaling infrastructure according to execution time rather than raw data volume, the platform supports real-time computation and provides uniform API access for consistent data delivery. This architecture is particularly well suited to scenarios where repeated analytical queries are performed over long time ranges.
Unifying access to weather data from several sources, at any point in the last 10 years, and across any location of the territory, is a challenging task to achieve with traditional n-tier technologies. This proof-of-concept demonstrated that our GreyCat technology can provide this service efficiently, sustainably, and can support more use cases involving this kind of geo-temporal data.
Dr. Grégory NAIN, Co-founder and Head of Operations, DataThings
Performance and efficiency
The impact of this new approach is substantial. As part of the proof-of-concept, DataThings carried out performance and cost estimations comparing the GreyCat-based approach with a traditional relational database setup under equivalent conditions.
According to these estimates, execution time was reduced from over one minute to approximately 90 milliseconds, while data transfer volumes decreased from 120 MB to 11 KB. As for the annual infrastructure costs, they were estimated to decrease from approximately €20,400 to €984.
Overall, this represents a 99.82% improvement in execution speed, 99.999% less data transfer, and a cost reduction of more than 94%.
Why It Matters
Efficient access to historical meteorological data enables a wide range of use cases, from operational decision-making, weather predictions, climate simulations and other research and innovation use cases.
By providing uniform access to the various data formats and resolutions collected from the measurement stations, we can facilitate the use of this data enormously, removing the repetitive and redundant data conversion steps for each data user. This capability can lead to improved use and accelerated access to this important measurement data, with significantly reduced IT resource consumption.
Bert Verdonck, CEO, LNDS
This collaboration provides a reference for how similar methods could be applied to other high-volume, time- and location-based datasets across the public sector.
Looking ahead
This proof-of-concept was conducted as an exploratory exercise to assess how alternative data architectures could ease the access to large-scale meteorological data. The insights gained continue to inform discussions around future data access patterns and infrastructure choices.
Potential next steps under consideration include extending similar approaches to restricted datasets, exploring the feasibility of a generic Luxembourg Weather Mesh API, and maintaining a low-code, cost-efficient model aligned with public-sector needs.
Engage with us
We continue to collaborate with public institutions, research organisations, and private-sector partners to support data-driven initiatives in Luxembourg. Exchanges around use cases, technical approaches, and lessons learned from exploratory collaborations such as this one help inform the development of future data services and support projects across different domains.