This is an old revision of the document!


Introduction

A lot of data comes in the form of time-series, and while usually they're easy to manipulate, the process can become tricky. Here TimEL comes to help!

Indeed TimEL is an excellent fit to solve various time-series problems, mainly the metering and billing ones (that's where it originates from).

Now let's image an hypothetical scenario to see how TimEL could help. Let's say you're implementing a next killer web game where you get a point for each rescued kittens. Now you want to praise your most loyal players with an in game bonus based on the achieved score.

More specifically, imagine that for each player session your backend system logs hourly the user session interval start-end and its relative score totalled during the session.

It might happen that sometimes a user session is still open, so in such a case the backend will emit a single entry covering more than hours when the session is complete.

Now imagine we have two players, A and B, with the following scores:

At 9:00, player A started to play and stopped at 11, scoring 36 points - so your backend logged 36 points in between 9:00 and 11:00. Player B started one hour later, and played only for one hour (scoring 20 points).

Now let's imagine that we want to compute the total score achieved by A and B together, every hour.

What would be the score in between Monday 12:00 and Tuesday 00:00 ? Well, it would be only A's one, but we have to consider that he saved 36 kittens for the whole 2 hours period.

Without more data, we can assume that if he saved 36 kittens in 2 hours, he saved 18 each hour. So we can say that between 09:00 and 11:00, 18 kittens were saved. One hour later 38 kittens were saved (18 by A, and 20 by B)!

Now let's compute the sum of the scores of A and B via TimEL:

Note that ki is a constant value in the range [ai, bi+1) and that the interval duration depends on the sampling frequency of the discretization process, but TimEL does not make any assumption regarding the duration of intervals: both constant-duration and irregular intervals are accepted.

The explicit reference to the interval underlying the validity of a value allows TimEL to clearly identify the undefined intervals - that is time intervals where the value is not available (or maybe just not available yet, for queue systems).

This approach is suitable to express a number of various processes, as example:

  • The value of a stock quote for a given period;
  • The consumption read by a meter reading for a given period;
  • An average speed for the given interval for a given period.

Below an example of TimEL code to evaluate A + B with the following values:

As you can see, TimEL will split A in the interval [April, October) as B has a value change in July, keeping thus the highest granularity possible resampling the input when needed.

Note that the resampling process behaves differently on the data type, for more information checkout Interpolation.

TimEL's expressivity comes mainly form the ability of model time recurrent events with a number of specific aggregation functions.

As the data type is very important since it affects the computation, multiple resample functions are available to support both rescale and data type conversion on the fly when needed.

The following basic function are available:

Each of these functions receive the value to resample as first argument, and can be used with an optional 2nd argument which is the temporal interval to which the resample should be applied.

Note that the temporal interval may be a fixed (defined using the Interval function) or a recurring one (defined using the Every function).

When no interval is provided, an aggregation function will produce a single output with the longest interval possible.

Follows an example of the following function, evaluated in the interval 2015-01-01 to 2015-02-01 (we use the ISO format to avoid confusion):

  • introduction.1556989618.txt.gz
  • Last modified: 2019/05/04 17:06
  • by a.leofreddi