Reason for the change

Our mission is to make high resolution energy data easily accessible to you. One of the ways we have been doing that is taking in files that are utility bills and converting them into data you can access via our API or Console. While doing this, a common piece of feedback we have been getting from you is that our current file statuses and status details (the File.fileStatus and File.fileDetails fields) are not informative enough for you to action upon what we tell you. We highly appreciate your feedback and in order to improve both clarity and transparency of our files processing pipeline, we will be making some updates to how we communicate our processing status of your files.

Change date

To be determined.

We are in the process of reaching out to you to share with you the details of this change and work with you to come up with a release date collaboratively in order to minimize the impact of this change to your systems. As soon as we have a release date, we will share it publicly here.

Before the change

Statuses

We use the File.fileStatus field to indicate the status of the file. This is a required field. This field will always be populated by Arcadia. The possible values are:

PENDING
SUCCESS
FAILURE

Status details

We use the File.fileDetails field to indicate the status detail of the file. This is an optional field. This will not always be populated by Arcadia. The possible values are:

New
In Progress
Provider Not Supported
Unsupported Provider
Under Review
Input File Corrupted
Unsupported Bill Format
Unsupported Language
File Processing Timed Out

All posible status transitions

PENDING → SUCCESS
PENDING → FAILURE

Webhooks behavior

When a file is uploaded and when the status of a file transitions, we will send a webhook with event type STATUS_CHANGE and subject type FILE. More details on webhook behaviors here.

After the change

Statuses

We will continue to use the File.fileStatus field to indicate the status of the file. This field will continue to be a required field. This field will always be populated by Arcadia. The new possible values are:

QUEUED
REJECTED
IN_PROGRESS
COVERAGE_REQUEST
SUCCESS

Status details

We will continue to use the File.fileDetails field to indicate the status detail of the file. This will now be a required field. This filed will always be populated by Arcadia. We will return as many reasons as we think applicable to the bill. The new possible values are:

READY_FOR_PROCESSING
PROVIDER_UNIDENTIFIABLE
SERVICE_TYPE_NOT_SUPPORTED
LANGUAGE_NOT_SUPPORTED
REQUIRED_DATA_FIELDS_MISSING
BILL_INSUFFICIENTLY_LEGIBLE
TIME_OUT
IDENTIFYING_PROVIDER
BILL_ANOMALY_FLAGGED
DATA_EXTRACTION_FAILURE_INVESTIGATION_STARTED
DATA_AUDIT_FAILURE_INVESTIGATION_STARTED
REQUEST_CREATED
IN_PROGRESS
DATA_EXTRACTION_SUCCESSFUL

Status and status detail definitions

File.fileStatus	File.fileDetails	Customer-facing definition
`QUEUED`	`READY_FOR_PROCESSING`	Arcadia has started processing the bill.
`REJECTED`	`PROVIDER_UNIDENTIFIABLE`	Arcadia is not able to obtain the provider name from the bill.
	`SERVICE_TYPE_NOT_SUPPORTED`	Arcadia does not currently support the service type of the bill.
	`LANGUAGE_NOT_SUPPORTED`	Arcadia does not currently support the language of the bill.
	`REQUIRED_DATA_FIELDS_MISSING`	Arcadia is not able to obtain one or more required fields from the bill.
	`BILL_INSUFFICIENTLY_LEGIBLE`	Arcadia is not able to read the necessary details of the content or structure of the bill.
	`TIMED_OUT`	Arcadia was not able to process the bill within 2 weeks, therefore the bill is timed out.
`IN_PROGRESS`	`IDENTIFYING_PROVIDER`	Arcadia was not able to automatically detect the provider. Our staff will try to identify the provider name from the bill.
	`BILL_ANOMALY_FLAGGED`	Arcadia has identified some anomalies on the bill and has started reviewing the bill to ensure accurate data extraction.
	`DATA_EXTRACTION_FAILURE_INVESTIGATION_STARTED`	The bill has failed automated data extraction. The bill will be reviewed for further processing.
	`DATA_AUDIT_FAILURE_INVESTIGATION_STARTED`	The bill has failed automated data quality audits. The bill will be reviewed for further processing.
`COVERAGE_REQUEST`	`REQUEST_CREATED`	Arcadia will develop an automated way to extract data from the bill.
	`IN_PROGRESS`	Arcadia has started the development process for automating data extraction for the bill.
`SUCCESS`	`DATA_EXTRACTION_SUCCESSFUL`	Arcadia was able to extract data from the bill submitted

All posible status transitions

Webhooks behavior

We will continue to use the same event type STATUS_CHANGE and subject type FILE as before. More details on webhook behaviors here.

Impacted customers

🚧
Are you currently using the files endpoints and webhooks?
If you are, in order for your integration to continue to work, you will need to make a few updates to how you handle file statuses and status details.
If you need more information, please reach out to your Customer Success Manager, Account Manager or our Support team.

If you are a Console-only user when it comes to uploading files and checking their statuses, this change will not break your integration. You will start to see the new statuses on Console when we release the change.

If you do not rely on the Files Module feature for your business, you can make the updates at a time that is most convenient for you.

Required customer actions

Update how you handle API responses from the files endpoints' File.fileStatus and File.fileDetails fields to handle the new statuses and status details.