Are there any troubleshooting tips for data refreshes?

Hello Eightfold Community,

Only datasets associated with a dashboard will be refreshed as part of the daily dataset refreshes. A user can:

• Monitor alert emails on dataset refreshes that are sent daily to confirm whether your dataset was refreshed or skipped or canceled. Look for emails with the subject [Airflow eu-central-1] quicksight_dataset_refresh.
  • There are two emails sent per day. One for the EU and one for the US West region. Click on the email based on the relevant region and search for customer name to identify which datasets were refreshed, skipped or canceled due to any errors.

If a dataset refresh was skipped since the dataset was not linked to an active dashboard, once a dashboard with the skipped dataset is provisioned for a customer, the dataset refresh will automatically pick up this dataset in the next refresh cycle. If however, there is a need to trigger a refresh before the daily dataset refresh use the steps below.

a. To trigger a dataset refresh use the following command:

python ./scripts/quicksight/refresh\_datasets.py --ds\_id ericsson\_user\_analytics\_with\_positions

b. Ensure that you see the following messages to confirm that the dataset was refreshed successfully:

I0819 16:33:19.906738 4264 refresh\_datasets.py:118] Tracking ingestion ericsson\_user\_analytics\_with\_positions\_26a357d5-fd6f-42c5-aa14-82d08b831372 for dataset ericsson\_user\_analytics\_with\_positions

I0819 16:34:21.162337 4264 refresh\_datasets.py:153] Ingestion ericsson\_user\_analytics\_with\_positions\_26a357d5-fd6f-42c5-aa14-82d08b831372 took 60 seconds and completed with status COMPLETED

Please ensure that the ingestion completes with the status COMPLETED which indicates successful ingestion.

c. If dataset ingestion fails for the dataset, use the following commands from AWS CLI to identify the error. Note: AWS CLI is only supported in us-west region and not in eu.

Here is the sample error output. If a dataset refresh fails due to an error, it will be stated in "ErrorInfo" in the response.

(py-virt) ec2-user@ip-172-31-27-151:~/vscode$ aws quicksight describe-ingestion --aws-account-id $AWS\_ACCOUNT\_ID --data-set-id airasia\_positions --ingestion-id airasia\_positions\_8581b0f3-e608-4599-83fd-b3e51a6aefd5

{

"Status": 200,

"Ingestion": {

"RequestType": "FULL\_REFRESH",

"RowInfo";: {

"RowsIngested": 7557,

"RowsDropped": 0

},

"RequestSource": "MANUAL",

"IngestionStatus": "COMPLETED",

"CreatedTime": 1629336483.326,

"IngestionSizeInBytes": 4600835,

"IngestionId": "airasia\_positions\_8581b0f3-e608-4599-83fd-b3e51a6aefd5",

"ErrorInfo": {},

"IngestionTimeInSeconds": 27,

"Arn": "arn:aws:quicksight:us-west-2:948299231917:dataset/airasia\_positions/ingestion/airasia\_positions\_8581b0f3-e608-4599-83fd-b3e51a6aefd5"

},

"RequestId": "762ed79b-e54d-4956-8c5d-0f2385f83cef"

}