Time, Calendar and periods

This section is a dive into three of the time constructs available from Shyft: the time type, the calendar, and the time period. ̨ It is necessary to understand the underlying time representation, and how to manipulate it, to be able to both display and work with time series data correctly, and to set up and perfom computations.

Contents:

To start of we need to import the time series library of Shyft, we put it into an alias sts.

import shyft.time_series as sts

Shyft time 

Shyft considers time as UTC+0 internally, is is stored and used as a number in the Shyft core and it can represent time at the microsecond resolution.

In the Python API time is exposed through the time class. Objects of this class is not that interesting, their main purpose is so the C++ implementation of Shyft can speak with the Python API correctly. In addition to this this class helps with parsing and stringifying UTC+0 time as well.

When using the time class is that values or objects of this class can both represent a concrete date/time, or a delta or offset without any particular meaning as a point in time. It is simply a value able to be interpreted in multiple ways.

We will mostly ignore the sub-second resolution of time.

The first way to initialize a time value is by passing it a number of seconds since the epoch. The epoch is the origin of the time system, Shyft uses the regular epoch: 1970-01-01T00:00:00.

time_point = sts.time(1559563200)
time_point # output: time(1559563200)

We can stringify the time value we just made. This will give us a time string following ISO8601:

str(time_point) # output: '2019-06-03T12:00:00Z'

We can also pass the ISO8601 string back to the time class to construct a moment as specified in the string.

Note that Shyft only parses a subset of ISO8601 so many perfectly valid time strings produces errors, and it does not even parse every time string it produces itself.

# mostly iso8601 time strings.
sts.time('2019-06-03T12:00:00.000000Z')  # output: time(1559563200)

An example of a ISO8601 that does not work is if we add the UTC-offset

try:
    sts.time('2019-06-03T11:00:00.000000+01')
except RuntimeError as e:
    print(e) # output: Needs format 'YYYY-MM-DDThh:mm:ssZ': got 2019-06-03T11:00:00.000000+01

Since time is a simple number we can work with it using simple arithmetics. We can add or subtract to it using + and -, multiply and divide it using * and /.

When doing arithmetics with time we can use regular numbers, the result will be a time object if at least one time object is involved in the computation. Numbers will be interpreted as seconds, while any fractional parts of the numbers will be interpreted as sub-seconds down to microseconds.

The next cell add one hour, 3600 seconds, to the time point we defined in the start of this section:

str(time_point + 3600) # output: '2019-06-03T13:00:00Z'

To get the current time we can use utctime_now() that we imported earlier. This function returns a time point representing the time at which the function was called:

str(sts.utctime_now()) # output: '2023-08-22T08:10:47.037387Z'

The three time constants we imported represent respectivly the least and largest time representable, and invalid time.

The minimum and maximum time constructs are mostly usefull as default values, and for creating time periods spanning “all” of time.

Invalid time may be returned by some functions to represent that whatever operation it were to do did not work, and some Shyft constructs initialize to it by default.

sts.min_utctime # output: time.min
sts.max_utctime # output: time.max
sts.no_utctime  # output: time.undefined

Calendars 

Since Shyft represent time solely as UTC+0 we need some construct that is able to work with localized time. This is the purpose of the calendar construct.

Initializing calendars 

To construct a Shyft calendar we need to provide the time zone specification it should use. If we do not specify any time zone it defaults to UTC+0.

utc = sts.Calendar()
utc_p5 = sts.Calendar('UTC+5')
utc_m2 = sts.Calendar('UTC-2')
oslo = sts.Calendar('Europe/Oslo')

We can list all the available specification by calling the function region_id_list on the calendar:

list(sts.Calendar.region_id_list())

Construct localized time 

Calendars can construct time points by using the time function on them. This function can not accept ISO8601 string, it instead accepts numbers specifying the time point.

The only required argument is the year, but it can accept as many as seven values, in order: year, month, day, hour, minute, second, microsecond. As long as we specify the year, we can specify as few or many of the other, but we always have to specify all values in the order listed.

utc.time(2019) # output: time(1546300800)

utc.time(2019, 5, 31, 13, 52, 30) # output: time(1559310750)

To demonstrate that calendars produce localized time, take a look at the two following cells. Notice that the same time on the same day is specified, but the results are not equal:

utc.time(2019, 5, 29, 12) # output: time(1559131200)

oslo.time(2019, 5, 29, 12) # output: time(1559124000)

Stringifying local time 

Calendars have a to_string function this function produces ISO8601 strings with offset like the calendar.

Note that Shyft (at the time of writing) cannot parse these strings again, as it only parses ISO8601 string where both the hour and minute of the offset is specified, but it writes strings with only the hour.

# stringify now using the local calendar
oslo.to_string(sts.utctime_now()) # output: '2023-08-22T10:26:53.434103+02'

Trim time 

Often when given an arbitrary time point we can trim it to move it back to a time point with some specific meaning like the beggining of the hour or week containing the time point. Take a look at the end of the calendar section to see the list of time constants available for trimming.

Trimming a time point is done using calendar semantics, so you can get different results by using different calendars.

# get the current time, then trim it to the start of the current hour
now = sts.utctime_now()
now_hour = oslo.trim(now, sts.Calendar.HOUR)

oslo.to_string(now_hour) # output: '2023-08-22T10:00:00+02'

# get the current time, then trim it to the start of the current hour
now = sts.utctime_now()
now_week = oslo.trim(now, sts.Calendar.WEEK)

oslo.to_string(now_week) # output: '2023-08-21T00:00:00+02'

Add to time 

If you want to add some offset to a time point that is not a specific length, like a month which could be 28, 29, 30, or 31 days, you should use the add function which handles these offsets easily.

The predefined offsets available for add is listed at the end of the calendar section.

The three next cells demonstrates the add function, first we retreive the current time and trim it to the start of the current month. The next two cells first add two months without calendar sematics and then with calendar semantics. Notice how you get diffenet results.

# get the current time, then trim it to the start of the current hour
now = sts.utctime_now()
now_month = oslo.trim(now, sts.Calendar.MONTH)

oslo.to_string(now_month) # output: '2023-08-01T00:00:00+02'

# add two months without calendar semantics
two_months = now_month + 2*sts.Calendar.MONTH

oslo.to_string(two_months) # output: '2023-09-30T00:00:00+02'

# add two months with calendar semantics
two_months = oslo.add(now_month, sts.Calendar.MONTH, 2)

oslo.to_string(two_months) # output: '2023-10-01T00:00:00+02'

Diff moments 

Using a calendar we can also get the number of times we can subdivide a timespan. Some timespans work with calendar semantics, like a month, or a year which can be off differing length. The predefined time spans that have calendar semantics is listed at the end of the calendar section.

When using the diff_units function we specify first the start and end time points, then the delta to subdivide with.

The function does not return any remainder, and only return the whole number of times the delta subdivied the period. You need check for a remainder manually if required.

# define start, end, and delta
t0 = utc.time(2018, 5, 5)
tN = utc.time(2018, 9, 17)
dt = sts.Calendar.WEEK

# get the number of times dt subdivides [t0, tN>
n_weeks = utc.diff_units(t0, tN, dt)
n_weeks # output: 19

Notice that the timespan we choose did not subdivide fully into weeks. If we add the number of weeks to the start point we do not get fully to the end point:

# add the computed number of weeks to the start point
utc.to_string(t0 + n_weeks*dt) # output: '2018-09-15T00:00:00Z'

Time constants 

The predefined time constants in Shyft is listed below. These are the ones for wich calendar semantics work.

Calendar.SECOND: One second
Calendar.MINUTE: One minute, 60 seconds
Calendar.HOUR: One hour, 3600 seconds
Calendar.WEEK: Defines a semantic week
Calendar.DAY: Defines a semantic day
Calendar.QUARTER: Defines a semantic quarter, 3 months
Calendar.MONTH: Defines a semantic month
Calendar.YEAR: Defines a semantic year

Time periods 

Shyft has a time period construct to represent a pair of time points representing a timespan. The type exposed through the Shyft Python API is the UtcPeriod construct.

The time period is represented as a half-open interval: The start value is included in the interval, while the end value is not included.

The two values of the period are time values, and thus represented as UTC+0.

Initialization 

A UtcPeriod is initialized from two time values or two numbers representing time:

sts.UtcPeriod(utc.time(2018), utc.time(2019)) # output: [2018-01-01T00:00:00Z,2019-01-01T00:00:00Z>

sts.UtcPeriod(1514764800, 1546300800) # output: [2018-01-01T00:00:00Z,2019-01-01T00:00:00Z>

List of functions 

UtcPeriod has several built-in functions and properties.

start
end
timespan
valid
to_string
trim
intersection
overlaps
contains
diff_units

Start & end 

start and end are properites which simply returns the start and end time points of the period.

period = sts.UtcPeriod(utc.time(2018), utc.time(2019))
period.start, period.end # output: (time(1514764800), time(1546300800))

Timespan 

timespan returns the $Delta t$ in seconds

period.timespan() # output: time(31536000)

Valid 

valid returns a boolean if the period is valid or not.

period.valid() # output: True
invalid_period = sts.UtcPeriod(utc.time(2019), utc.time(2018))
invalid_period.valid() # output: False

To string 

to_string returns the period start and end in ISO8601 format with the half-open interval symbols as a string.

period.to_string() # output: '[2018-01-01T00:00:00Z,2019-01-01T00:00:00Z>'

Trimming periods 

As with time values, time periods can also be trimmed to multiples of specific time deltas. When you have a UtcPeriod object you call the trim function on it to do this.

The time deltas are the same ones listed at the end of the calendar section.

When trimming time periods we have two modes to choose between. We can trim outwards to a multiple of the delta, or inwards to a multiple of the delta. This is specified as the third argument to the function. If it is left unspecified the function defaults to trimming inwards.

The first argument to the function is the calendar to use when trimming, the second argument is the delta to trim to, and the optional third argument is the trim policy.

period = sts.UtcPeriod(utc.time(2017, 6), utc.time(2019, 6))
period.trim(utc, sts.Calendar.YEAR, sts.trim_policy.TRIM_IN)
# output: [2018-01-01T00:00:00Z,2019-01-01T00:00:00Z>

period = sts.UtcPeriod(utc.time(2017, 6), utc.time(2019, 6))
period.trim(utc, sts.Calendar.YEAR, sts.trim_policy.TRIM_OUT)
# output: [2017-01-01T00:00:00Z,2020-01-01T00:00:00Z>

Intersection & overlap 

The time period construct have the functions to determine overlap between periods, and compute the intersection between two periods.

To demonstrate we define three periods, one for the first three month of the year, one for the middle three, and one for the last three:

period_1 = sts.UtcPeriod(utc.time(2018, 1), utc.time(2018, 7))
period_2 = sts.UtcPeriod(utc.time(2018, 4), utc.time(2018, 10))
period_3 = sts.UtcPeriod(utc.time(2018, 7), utc.time(2019, 1))

To determine if they overlap we can call the overlaps function on one of the periods, and pass the other as argument:

period_1.overlaps(period_2) # output: True

period_1.overlaps(period_3) # output: False

The intersection is not implemented as a function in the UtcPeriod structure, it is a free function. This is why we had to implement it separatly.

To compute the intersection we pass the intersection function both the periods to compute intersection for. We get a special period out if the periods does not overlap.

sts.intersection(period_1, period_2) # output: [2018-04-01T00:00:00Z,2018-07-01T00:00:00Z>
sts.intersection(period_1, period_3) # output: [not-valid-period>

Moment in period 

To check if a time point is inside a period we can call the contains function on the period with the time point as argument.

period = sts.UtcPeriod(utc.time(2018), utc.time(2019))

The contains function accepts the time point, and returns a boolean signifying whether the point is within the period or not:

moment = utc.time(2017, 6)
period.contains(moment) # output: False

moment = utc.time(2018, 6)
period.contains(moment) # output: True

Diff a period 

To determine how many times a period can be subdivided by a delta, we do not need to unpack the points defining the period and use Calendar.diff_units. The UtcPeriod construct have a diff_units function accepting a calendar and the delta. Otherwise this function works just like the Calendar.diff_units.

To demonstrate we define a period of some length:

# define a period
period = sts.UtcPeriod(utc.time(2018, 3, 15, 17), utc.time(2018, 10, 2, 4))

By using UtcPeriod.diff_units on the period with Calendar.DAY as the delta, we get the number of whole days in the period:

# determine the number of whole days the period can be subdivided into
period.diff_units(utc, sts.Calendar.DAY) # output: 200

Or we can determine the number of minutes the period can be subdivided into:

# determine the number of whole minutes the period can be subdivided into
period.diff_units(utc, sts.Calendar.MINUTE) # output: 288660