Files Processing
An overview of how Arcadia accesses utility data from uploaded bills and invoices along with the coverage we support.
Overview
The primary way for Arcadia to access utility data is by obtaining credentials for online utility provider accounts. When that is not a viable option, Arcadia can obtain data from bills directly and deliver that data to you via the same delivery mechanisms.
You can transfer bills to Arcadia via the following methods:
- API: upload one or more files via the files endpoints.
- Console: upload one or more files via the Console UI.
After Arcadia processes a file successfully, the corresponding statement data will be available via both API and Console.
You can view all of your uploaded files on the “Files” tab in the Dashboard or get the same data via API.
File Upload Limitations
Today, file uploading is limited to a maximum of 75 files in one upload and a total upload size (across all files) of 1GB.
How file processing works
At a high level, each file Arcadia receives goes through the following steps:
- PDF type identification
We identify whether the file you’ve uploaded is a selectable PDF or not. A selectable PDF is a PDF file that contains text that is selectable by a cursor. - Automated or assisted automated provider detection
We identify which utility provider is represented on the statement. - Data extraction
We extract the data from the bill, normalize the data, and prepare it for data quality auditing. - Data quality audits
We run over a hundred data quality checks on the data and prepare it for delivery. After a file passes this step, you will be able to access the corresponding statement data via API or Console.
When the file is from a provider we do not support today, we will attempt to build coverage for that provider before extracting the data. Timelines for new provider coverage builds may vary and we do not support all global languages.
Coverage expansion requests
We recommend you work closely with your Arcadia onboarding team to discuss coverage expansion requests and to monitor progress.
Coverage we support today
The following diagram outlines what files can be supported by Arcadia today.
Files Processing Coverage Diagram
Provider and landlord bills support
Arcadia supports processing bills that are issued by utility providers or landlords. In order for Arcadia to accept landlord bills for processing, the bill must be in English and in English only.
Here are the required data points that needs to be present on the bill:
| Required Data Point | Description |
|---|---|
| Name of the Utility Provider / Landlord / 3rd party originator | Normally this is the utility provider. For leased properties, this could be the party that originated the bill e.g. the landlord or a 3rd party. |
| Service Address | Preferably, the full address of the location at which the services are being provided i.e. street address, city, state/province, zip/postal code, and country. At a minimum, street address and building number needs to be available. |
| Account number or similar unique identifier | We require unique identifier to track invoices month to month. In some cases, if the service address on the bill is unique, it can be used as an account number |
| Start Date / End Date / Invoice Date | Dates to identify the billing interval or consumption period. At least one date should be available. If there is only a single date, Arcadia will assume a period of 30 days from that date. |
| Consumption | Amount or quantity of commodity consumed. There should be a target for the full quantity used in the invoice period; hourly or daily breakdowns are not acceptable |
| Unit of Measure | Unit of Measure of consumption (kWh, etc.). |
Don't have access to required data points?
If the bills don’t include the data points listed above, please work with your assigned Arcadia customer success team to determine alternative methods of capturing utility data for your accounts.
Optionally, you can also provide the following data points on the bill:
| Optional Data Point | Description |
|---|---|
| Commodity Type | Type of service i.e. Electric, Gas or Water. If not present, at least the Unit of Measure needs to be available |
| Total Cost / Bill Amount and Currency | Required only if customer requires cost and charges to be captured from the bill |
Where should these data points be?
These data points must be be present on the bill - not in the file name or elsewhere.
OCR support
When the content on the bill cannot be selected by a cursor, Arcadia will first attempt to perform OCR on the bill image before proceeding to the data extraction step. Currently, Arcadia supports OCR in the following languages:
- English
- French
- Spanish
- German
- Chinese Simplified
- Dutch
- Finnish
- Greek
- Italian
- Turkish
- Slovenian
- Polish
- Czech
- Swedish
- European Portuguese
- Japanese
- Quechua
- Aymara
- Korean
- Maori
- Slovak
- Romanian
- Norwegian Bokmal
- Arabic
- Hebrew
- Croatian
- Vietnamese
If the language of the unselectable PDF bill is not among the above languages, please raise a support ticket and request the language to be supported along with a sample bill. We will review the information provided and share the feasibility of the request on the ticket.
Supporting OCR in a language does not mean we are able to support data extraction from that provider.
The OCR step converts an image into a selectable PDF. After the conversion is successful, we will attempt to identify the provider of the bill. If we are able to identify the provider of the bill and the provider supported by Arcadia (call the GET List Providers endpoint for the full list of supported providers), then we will be able to perform data extraction.
If the provider is not among the supported list of providers, please raise a support ticket or work with your Arcadia onboarding team directly to request the provider to be supported.
What happens when OCR fails due to poor image quality?
When we are unable to perform OCR due to poor image quality, the uploaded bill will be
REJECTEDwith the status detailBILL_INSUFFICINETLY_LEGIBLE.
Provider support
For a full list of providers we support, you can obtain that from the “Providers” tab on Console or via the providers endpoint.
Provider coverage extension support
When the provider of the bill is not currently supported by Arcadia, we will explore extending provider coverage support for that provider. Please contact your Arcadia onboarding team to discuss timelines for new coverage requests. We currently support extending coverage if a bill is in one of the following languages:
- English
- English bills with Ghanaian Celi as currency
- Dutch
- Finnish
- French
- German
- Italian
- Polish
- Romanian
- Spanish
- Indonesian
- Swedish
- (Brazilian) Portuguese
- (European) Portuguese
- Slovak
- Slovenian
- Danish
- Czech
- Malay
- Turkish
- Hindi
- Arabic
- Bomkal
- Greek
- Thai
- Vietnamese
Updated 5 months ago
Check out how Arcadia will communicate the processing status of each file to you in the next page.