feat: store query context when saving charts

[betodealmeida]

SUMMARY

This PR adds a new column to the slices table called query_context. It also modifies Explore to save the query context to this column whenever a chart is saved.

Having the query context is needed in order to generate data for a saved chart with the api/v1/char/data endpoint. Since the query context is built by each visualization from slices.params via Javascript, there's currently no way to fetch data for a given chart from Python.

With the new column we will be able to generate CSV reports for new (JS-only) charts, like the pivot table v2.

The migration script only adds the column, but doesn't populate it for existing charts since that would be too expensive and complex.

Migration benchmark:

Results:

Current: 0.26 s
10+: 0.35 s
100+: 0.30 s
1000+: 0.22 s
10000+: 0.22 s

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

Save a chart and check the slices.query_context column:

mysql> SELECT params, query_context FROM slices WHERE id=143\G
*************************** 1. row ***************************
params: {
  "adhoc_filters": [],
  "aggregateFunction": "Sum",
  "colOrder": "key_a_to_z",
  "colTotals": true,
  "datasource": "3__table",
  "extra_form_data": {},
  "granularity_sqla": "ds",
  "groupbyColumns": [
    "state"
  ],
  "groupbyRows": [
    "name"
  ],
  "metrics": [
    {
      "aggregate": "SUM",
      "column": {
        "column_name": "num",
        "type": "BIGINT"
      },
      "expressionType": "SIMPLE",
      "label": "Births",
      "optionName": "metric_11"
    }
  ],
  "metricsLayout": "COLUMNS",
  "rowOrder": "key_a_to_z",
  "rowTotals": true,
  "row_limit": 50000,
  "slice_id": 143,
  "tableRenderer": "Table With Subtotal",
  "time_grain_sqla": "P1D",
  "time_range": "100 years ago : now",
  "time_range_endpoints": [
    "inclusive",
    "exclusive"
  ],
  "url_params": {},
  "valueFormat": "SMART_NUMBER",
  "viz_type": "pivot_table_v2"
}
query_context: {
  "datasource": {
    "id": 3,
    "type": "table"
  },
  "force": false,
  "queries": [
    {
      "time_range": "100 years ago : now",
      "granularity": "ds",
      "filters": [],
      "extras": {
        "time_grain_sqla": "P1D",
        "time_range_endpoints": [
          "inclusive",
          "exclusive"
        ],
        "having": "",
        "having_druid": [],
        "where": ""
      },
      "applied_time_extras": {},
      "columns": [
        "state",
        "name"
      ],
      "metrics": [
        {
          "aggregate": "SUM",
          "column": {
            "column_name": "num",
            "type": "BIGINT"
          },
          "expressionType": "SIMPLE",
          "label": "Births",
          "optionName": "metric_11"
        }
      ],
      "annotation_layers": [],
      "row_limit": 50000,
      "timeseries_limit": 0,
      "order_desc": true,
      "url_params": {},
      "custom_params": {},
      "custom_form_data": {}
    }
  ],
  "result_format": "json",
  "result_type": "full"
}
1 row in set (0.00 sec)

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Merging #15824 (f922068) into master (32a5680) will decrease coverage by 0.24%.
The diff coverage is 90.47%.

❗ Current head f922068 differs from pull request most recent head 6797fad. Consider uploading reports for the commit 6797fad to get more accurate results

@@            Coverage Diff             @@
##           master   #15824      +/-   ##
==========================================
- Coverage   77.14%   76.90%   -0.25%     
==========================================
  Files         984      984              
  Lines       51662    51670       +8     
  Branches     6991     6991              
==========================================
- Hits        39855    39736     -119     
- Misses      11583    11710     +127     
  Partials      224      224              

Flag	Coverage Δ
hive	?
javascript	71.81% <0.00%> (-0.02%)	⬇️
mysql	81.54% <100.00%> (-0.03%)	⬇️
postgres	81.56% <100.00%> (-0.03%)	⬇️
presto	?
python	81.65% <100.00%> (-0.46%)	⬇️
sqlite	81.19% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...t-frontend/src/explore/actions/saveModalActions.js	59.37% <0.00%> (-1.92%)	⬇️
superset/common/query_object.py	90.14% <ø> (ø)
superset/databases/commands/validate.py	74.57% <ø> (ø)
superset/charts/schemas.py	100.00% <100.00%> (ø)
superset/connectors/sqla/models.py	88.31% <100.00%> (-1.64%)	⬇️
superset/db_engine_specs/base.py	88.28% <100.00%> (-0.40%)	⬇️
superset/db_engine_specs/mysql.py	97.61% <100.00%> (ø)
superset/db_engine_specs/postgres.py	97.36% <100.00%> (ø)
superset/db_engine_specs/presto.py	83.36% <100.00%> (-6.53%)	⬇️
superset/initialization/__init__.py	87.91% <100.00%> (ø)
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32a5680...6797fad. Read the comment docs.