Arc Data Model
Learn how data is structure, how to find the data you need, and handle edge cases.
Get to know the Arc Data Model before you start using the Dashboard or API
This guide will provide a high level orientation for how data obtained from utility bills is structured. Familiarizing yourself with the Data Model before getting started with the Plug Statements product can help you fine the data most relevant to your use case faster.
Entities in the Data Model
| Entity | Description |
|---|---|
| Provider | An entity that provides a commodity to a market as a service. Often referred to as a "utility" in some contexts. See Service Types Guide for a list of commodities that Arcadia supports. |
| Account | A way to track and manage the usage of various utility services provided to a residential or commercial property. Utility accounts are mostly for billing purposes, as they allow the provider to bill customers for the services they have consumed in a billing period. |
| Credentials | The set of information (often, but not always and not limited to, username/email and password) used to access an online Account on a Provider’s website or data portal. You can use a correlationId to annotate Credentials with an identifier that helps you to correlate it with records in your system. |
| Statement | The processed, standardized, machine-readable entry within the platform that represents the content of a bill issued by a Provider according to Arcadia's data model. |
| Meter | A meter in Arcadia's data model can be either: - A physical monitoring device located at the point of commodity consumption that reports consumption in regular intervals for the provider to calculate charges. - A proxy by Arcadia to represent consumption at the most granular level with the data model. |
| Site | A concept in Arcadia's data model to enable grouping meters by a location or another logical grouping for your system. Using this concept is optional. |
| Tariff | The set of rules Providers use to calculate and distribute charges for commodities with measured, estimated, and billed usages. A meter can only have one tariff at any given time. Even though the tariff of a meter can change over time, at any given time, it can only have one tariff. |
Provider Classifications
"Provider Classifications" are designations determined by the provider. Whether a Provider is a Publisher or Pass-Through provider depends on the Account in the context of a particular bill. All statements will have an account that is from a PUBLISHER provider. Not all statements will have an account that is from a PASS_THROUGH provider.
| Entity | Description |
|---|---|
| Publisher | A utility in a deregulated, open-competition, market that serves as the statement publisher (i.e. issuer) of the bill. |
| Pass-Through | A utility in a deregulated, open-competition, market that serves as a pass-thru utility mentioned within a statement published by another utility (i.e. the Publisher Provider). |
Account Types
"Account Types" is a designation determined by Arcadia. The type of the account is inferred by Arcadia based on what information is available on the bill. When we see a summary account number present on the bill, we infer that the account is a SUMMARY account type. SUMMARY and SUB_ACCOUNT types typically only exist for commercial accounts. When an account is not a SUMMARY or SUB_ACCOUNT, it is a NORMAL account.
| Entity | Description |
|---|---|
| Normal | The most common subtype of an Account. It is a single Account that has no sub-accounts. This is not a provider-determined account type. |
| Summary Account | A sub-type of Account which is used to tally up all the charges across all the Sub-accounts owned by the same Account holder. |
| Sub-Account | A sub-type of Account that has a main tracking account (i.e. a Summary Account). |
Possible Data Model Scenarios
Here are some common ways we model the utility data that we obtain from providers. Though these scenarios are not exhaustive, they are representative of how we present most bills from a US provider.
Scenario 1: When an account is a NORMAL account
NORMAL account
Scenario 2: When accounts from PUBLISHER and PASS_THROUGH providers are present on one statement.
PUBLISHER and PASS_THROUGH providers are present on one statement.
Locating the data you need
The data you need are often located in a charge or usage object. For example, if you are looking for cost line items, they are most likely in a charge object; if you are looking for the amount of a commodity consumed (e.g. amount of electricity used), you can find that in a usage object.
charge and usage objects can occur at any of the three levels:
- Statement
- Account
- Meter
The scenarios below illustrate a few common cases for how charge and usage level data can be located on a statement graph. Familiarizing yourself with these will help you traverse the Detailed Statement response more easily and find what you need.
Scenario 1: When the statement has one account and one meter.
Scenario 2: When the statement has one account with multiple meters.
Charges and usages often live at both the account and meter level.
This is scenarios is most often, though not exclusively, seen when the service type is Gas.
How the Proration and Inference (P&I) feature works
For more details on how the Proration and Inference feature works, please refer to this Guide.
In this illustrative example, the statement has charges and usages at the account level - two charges and two usages. It does not have any charges and usages at the meter level.
Data model before P&I
Before P&I, the placement of the data would be exactly as how it is presented on the bill.
Data model after P&I
After P&I, account level charges and usages will be represented at the meter level as well.
Handling Common Data Scenarios and Edge Cases
| Scenario | Our Recommendation | Rationale |
|---|---|---|
The classification field in the provider object embedded in the accountData object on a GET Statement response populated as PUBLISHER or PASS_THROUGH. | If your use case is able to tolerate an inaccuracy of up to 1%, use the charge and usage values from the PUBLISHER provider and ignore those from the PASS_THROUGH provider.If your use case requires 100% accuracy compared to the bill issued by the provider (e.g. Bill Pay use cases), you should take into account values from both provider classifications. ⚠️Please be aware that taking this approach requires in-depth understanding of our data model and extensive additional engineering work on your side. | Upon validating a representative set of bills for all US providers, we conclude that whether you take into account the values from the PASS_THROUGH providers only contributes to a deviation from the actual values by less than 1% .Therefore, unless there is significant business value for your use case, we do not recommend using values from PASS_THROUGH providers. |
PASS_THROUGH providers may display different usage amounts than the PUBLISHER Provider from the same invoice. | (Same as above) | (Same as above) |
TheusageUnit and/or the totalUsageUnit being returned as null or Undefined. | Treat it as the provider did not publish these on the bill. | Upon extensive validation work, we are confident that our data extraction abilities to pick up relevant units is functioning as expected for all US providers. If you wish us to investigate a specific case further, please feel free to raise a Support ticket. However, you may experience prolonged response times for these issues as they require extensive work from our teams. |
| Some electric or water invoices with charges, have no usage. | (Same as above) | (Same as above) |
The statementDate field sometimes returns NULL. | (Same as above) | (Same as above) |
The meterNumber field (and subsequently the normalizedMeterNumber field) sometimes returns NULL. | (Same as above) | (Same as above) |
LIGHTING and ELECTRIC are two distinct service types. | Consider both service types when calculating total usages/costs. (📑 Refer to our Supported Service Types Guide for more information) | - |
The service type SEWER cannot exist without meters for the service type WATER meters | Consider both service types when calculating total usages/costs. (📑 Refer to our Supported Service Types Guide for more information) | - |
| "On peak charges" and "super peak charges" are not the same. | - | Please note that these are terms for distinct time of uses. |
| Not all multi-meter statements have synchronized reads. | When there are more than one meter on a statement, please use the periodStart and periodEnd dates at the meter-level, not the ones at the account-level. | - |
A statement with many (e.g. 30+) meters is not an invoice from a SUMMARY account. | Whether an account is a SUMMARY account is determined by the provider.If the provider does not regard an account as a SUMMARY account, how the account is structured or the number of meters associated with the account does not change it's Account.type designation. | - |
Updated 8 months ago