Data Replication
You may want to take Samsara's data and replicate it into a more centralize database (e.g. Azure SQL DB) or a data warehouse (e.g. Databricks, Snowflake). This is often done to remove your data silos and consolidate your data from several systems into one place, which simplifies how your organization can leverage data to make better decisions or build new custom technologies.
There are two ways to do Data Replication today:
- Build your own batch processing job using Samsara's Open API
- Use a data connector from our App Marketplace that can replicate data with low-code/no-code required. Our list of data connectors includes Fivetran, Zapier, Precog, and Tray.io
- If you need assistance with building a custom connector, you can review a list of certified Samsara experts.
- If you are looking to access Samsara data in Power BI, an alternative path is to leverage a Power BI connector, which directly connects to Samsara's API vs replicating the data into a separate data storage. See our Power BI Connector guide for more.
The rest of this guide will focus on the "build it yourself" option
Step 1: Identify Data Types to Replicate
At Samsara, we have 2 main types of data that can be replicated:
Data Type | Examples |
---|---|
Core Objects | These are objects that are central to how Samsara's platform is configured and used. These can be created, edited, viewed, and deleted by user input - whether in our Samsara dashboard or Samsara mobile apps. Examples: - Addresses - Tags - Users - Drivers - Vehicles - Trailers - Equipment - Routes |
Time Series Objects | This is time-based data that Samsara collects from the real-world with our hardware devices or generates as part of our software offerings. This type of data can be leveraged for powerful analytical insights into your organization. For example, replicate all safety events and deeply analyze safety behavior. Or, replicate all driving log data and evaluate driving patterns by region that your fleet has. Examples: - Vehicle GPS location - Vehicle odometer - Vehicle fault codes - Vehicle engine states - Vehicle aux input values - Vehicle battery levels - Vehicle DEF levels - Driver-vehicle assignments - Safety events - Driver HOS logs - Driver app sign-in/sign-out - Route Stop audit-logs |
Fundamentals of Data Replication
Samsara recommends using the Samsara APIs in order to accomplish this. Our API endpoints have a wide coverage across all types of data at Samsara, are reliable, and are performant.
If there is feedback on our API endpoints not being conducive to data replication, or feedback on other methods to accomplish data replication, please fill out this feedback form.
If you are looking for technical assistance on how to accomplish this, you can post in our Discussion Forum or you can engage one of our certified integration experts here.
Replicating Core Objects
There is a general pattern for replicating core object data from Samsara via API
- Step 1 - Backfilling: Iterate through all objects and add them as records to a target table
- Step 2 - Consuming New Data and Changes: Every 24 hours, iterate through all objects that have been added or changed in the last 24 hours, and create a new record or update an existing record
- Step 3 - Remove Deleted Objects (Optional): Every 24 hours consume if there were any objects deleted and remove those from the table
Replicating Time Series Objects
Similar to core objects, there is a general pattern for replicating core object data from Samsara via API
- Step 1 - Backfilling: Iterate through all objects and add them as records to a target table
- Step 2 - Consuming New Data and Changes: Every 24 hours, iterate through all objects that that have been added or changed in the last 24 hours, and create a new record or update an existing record
- Step 3 - Handling Dense Data (Optional): Some types of time series data are very dense (e.g. GPS data from vehicles, or odometer values from vehicles). In these cases, it may make sense to parallelize your API calls in case processing time is too high for your goals (e.g. you want to replicate data more real-time). For guidance on how to do this, review this code sample.
Aggregate Metrics
Samsara has a number of aggregate metrics displayed in our software products (e.g. Idle time over the last week, a driver's safety score over the last month, time-on-site for a location over the last day). While some of these data points are available via API, we encourage using the raw time series data we expose to calculate these metrics yourselves. This also unlocks opportunities for you to tweak definitions of aggregate metrics in favor of how your organization operates.
Updated 9 months ago