Demystifying Time Series: A Beginner’s Guide to dealing with Date & Time (Part 1)

Image Source — https://unsplash.com/photos/BXOXnQ26B7o

Introduction

Time Series Analysis is one such field in Data Science that is of utmost importance in the industry. From manufacturing to finance, various domains deal with time-series data.

Going forward, I would be covering a ton of time-series posts ranging from analyzing the data to building models. Those would be completely hands-on blogs using Python on Jupyter Notebook.

I would be leveraging Solar Data from Kaggle to showcase some of the fundamental operations you could perform on a DateTime field

Solar generation and demand Italy 2015-2016

Basic Operations

Now, lets see what we could do with time-series data -

1. The Timestamp is in UTC format as well it is object type

data.dtypes

2. Convert UTC_TIMESTAMP field from object type to datetime and store in a new column TIMESTAMP

# Convert UTC_TIMESTAMP from object type to datetime
from datetime import datetime, timedelta
data['TIMESTAMP'] = pd.to_datetime(data['UTC_TIMESTAMP'], format = "%Y-%m-%d %H:%M:%S")

3. Convert TIMESTAMP from datetime64[ns, UTC] to datetime64[ns]

# Convert TIMESTAMP from datetime64[ns, UTC] to datetime64[ns]
data['TIMESTAMP'] = data['TIMESTAMP'].dt.tz_localize(None)

4. Fetch Day, Month and Year

# Day
data['DAY'] = data['TIMESTAMP'].dt.day
# Month
data['MONTH'] = data['TIMESTAMP'].dt.month
# Year
data['YEAR'] = data['TIMESTAMP'].dt.year

5. Fetch Hour, Minutes and Seconds

# Hour
data['HOUR'] = data['TIMESTAMP'].dt.hour
# Minutes
data['MINUTE'] = data['TIMESTAMP'].dt.minute
# Seconds
data['SECOND'] = data['TIMESTAMP'].dt.second

6. Fetch Day of Week and Month Name

# Day of week
data['DAY_OF_WEEK'] = data['TIMESTAMP'].dt.day_name()
# Name of the Month
data['NAME_OF_MONTH'] = data['TIMESTAMP'].dt.month_name()

7. Fetch Quarter

# Get Quarter
data['QUARTER'] = data['TIMESTAMP'].dt.quarter

8. Is Month Start or Month End

# Is Month Start
data['IS_MONTH_START'] = data['TIMESTAMP'].dt.is_month_start
# Is Month End
data['IS_MONTH_END'] = data['TIMESTAMP'].dt.is_month_end

9. Is Quarter Start or Quarter End

# Is Quarter Start
data['IS_QUARTER_START'] = data['TIMESTAMP'].dt.is_quarter_start
#Is Quarter End
data['IS_QUARTER_END'] = data['TIMESTAMP'].dt.is_quarter_end

10. Is Leap Year

# Is Leap Year
data['IS_LEAP_YEAR'] = data['TIMESTAMP'].dt.is_leap_year

Conclusion

Time Series is a broad field where you could do so much but still fall short of achieving the end goal. This blog post is a beginner friendly guide to some key operations you could perform in a time series data as part of your EDA pipeline.

In the future blog posts, I could be covering topics like resampling, date range, and so on. This is my first post and any feedback would be helpful down the line.