This is an old revision of the document!


TimEL: Time-series Expression Language

Pick a random example in the 📺 online console and try it yourself!

Manipulating time-series is a common use case for metering, monitoring, IoT and pay-per-use billing systems.

You may wonder: there are other time-series libraries out there, why TimEL?

Keep your data: do not resample your time series

Many time-series systems automatically upscale your value to a certain interval (let's say a value every 60 seconds): this produces a lot of duplicates and also limits granularity: what if there is a value change that lasted for less than 60 seconds? The information will be somehow lost, as it will be blended with some other value to accommodate the 60 second boundary constraint.

TimEL won't force you to do that: it will operate directly using your sample timestamps, regardless of their form (aligned, misaligned, constant interval or variable one) or granularity (anything from years to milliseconds). The granularity of your samples drives the computation, not the other way around!

Do the right thing: gauges vs integrals (counters)

Some time-series data come as a gauge, while some other come as a counter (integral). TimEL encodes this information directly within its typing system, so you do not have to worry.

Let's imagine you're summing two integral (counter) values: if they're not perfectly aligned, TimEL will automatically interpolate when needed.

TimEL is a Java library to compile and evaluate TimEL expressions. TimEL expressions are written in a user-friendly language that allows time-series manipulation without the need of taking care about upscaling, downscaling or mixing up different time series intervals.

Let's see an expression to count the number of days:

scale(                                      // (3) and then downsample for the whole interval
    scale(
        uniform(1.0),                       // (1) let's take an integral value 1.0
        every(1, "DAY_OF_YEAR", "UTC")      // (2) repeat it every day
    )
)

If we evaluate this expression for an interval in the same day, let's say 06:00-18:00, it'll report 0.5 - that is half day. If we evaluate it for more days it will count how many days are contained in the interval. The function uniform here returns an integral, so TimEL knows how to interpolate it properly - that is handled by the interpreter so the user does not need to worry no more about time frames.

With TimEL you can:

  • Mix multiple time frames - for example you can sum daily data with hourly data, or even non-regular data like monthly data;
  • Express easily recurrent quantities, like 10 units every hour;
  • Scale natively integral values (like consumptions) and averages;
  • Stream results without the need of having all the operands in memory;
  • Support integer, floating point and double expressions;
  • Extend with your own types and functions;
  • Evaluate expression securely - by default there is no way to access external JVM objects or methods that would expose you to a security risk.

TimEL requires Java 8 and will run in any J2SE or J2EE container.


Acknowledgements

I wish to thank ej-technologies for providing the TimEL development team with a free JProfiler license, a great Java Profiler.

  • home.1624438059.txt.gz
  • Last modified: 2021/06/23 08:47
  • by a.leofreddi