Skip to content

feat: call screenshot to store query_context#15846

Merged
betodealmeida merged 6 commits intoapache:masterfrom
betodealmeida:report_populate_query_context
Jul 27, 2021
Merged

feat: call screenshot to store query_context#15846
betodealmeida merged 6 commits intoapache:masterfrom
betodealmeida:report_populate_query_context

Conversation

@betodealmeida
Copy link
Copy Markdown
Member

@betodealmeida betodealmeida commented Jul 22, 2021

SUMMARY

#15830 modified the _get_csv_data functionality in the reports executor to use a new chart API introduced in #15827. One problem is that the new API can only be called for charts that were saved with their query_context, a change that was introduced in #15824.

Pre-existing charts have query_context set to null, so we can't use the new API with them. As a workaround, @hughhhh implemented logic to save query_context when a chart that doesn't have it saved is loaded (#15865). This PR leverages that work by taking a screenshot of the chart using Selenium whenever we need to call the CSV API but query_context is null.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

  1. Create a chart, save.
  2. Delete query_context of the chart (UPDATE slices SET query_context = NULL WHERE id=XXX)
  3. Create a CSV report based on the chart
  4. Report with CSV should arrive correctly, and query_context should be back.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@betodealmeida betodealmeida requested review from eschutho, hughhhh, lilykuang and villebro and removed request for lilykuang July 22, 2021 22:13
@pull-request-size pull-request-size Bot added size/M and removed size/S labels Jul 26, 2021
@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 26, 2021

Codecov Report

Merging #15846 (1a2d399) into master (62550db) will decrease coverage by 0.29%.
The diff coverage is 41.17%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15846      +/-   ##
==========================================
- Coverage   77.05%   76.76%   -0.30%     
==========================================
  Files         986      986              
  Lines       51944    51993      +49     
  Branches     7081     7090       +9     
==========================================
- Hits        40025    39911     -114     
- Misses      11693    11856     +163     
  Partials      226      226              
Flag Coverage Δ
hive ?
javascript 71.69% <11.11%> (-0.10%) ⬇️
mysql 81.39% <75.00%> (-0.05%) ⬇️
postgres 81.42% <75.00%> (-0.05%) ⬇️
presto ?
python 81.51% <75.00%> (-0.48%) ⬇️
sqlite 81.05% <75.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ntend/src/explore/components/ExploreChartPanel.jsx 15.00% <0.00%> (-1.67%) ⬇️
superset/models/slice.py 86.18% <ø> (ø)
superset/reports/commands/execute.py 92.17% <75.00%> (-0.72%) ⬇️
...d/src/explore/components/PropertiesModal/index.tsx 81.94% <100.00%> (+1.68%) ⬆️
superset/db_engines/hive.py 0.00% <0.00%> (-82.15%) ⬇️
superset/db_engine_specs/hive.py 69.80% <0.00%> (-16.87%) ⬇️
superset/db_engine_specs/presto.py 83.47% <0.00%> (-6.49%) ⬇️
...-frontend/src/dashboard/reducers/dashboardState.js 66.66% <0.00%> (-4.31%) ⬇️
...perset-frontend/src/addSlice/AddSliceContainer.tsx 78.12% <0.00%> (-3.93%) ⬇️
.../src/explore/components/ControlPanelsContainer.tsx 76.02% <0.00%> (-3.00%) ⬇️
... and 26 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62550db...1a2d399. Read the comment docs.

@betodealmeida betodealmeida force-pushed the report_populate_query_context branch from 869cfa4 to c089468 Compare July 26, 2021 20:20
@pull-request-size pull-request-size Bot added size/L and removed size/M labels Jul 26, 2021
Comment thread superset-frontend/src/explore/components/PropertiesModal/index.tsx
Comment thread superset/reports/commands/execute.py Outdated
# get a screenshot to force the chart to produce and save the query
# context.
if self._report_schedule.chart.query_context is None:
self._get_screenshot()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think there may be a case where the screenshot finishes before the put? Do we need some error handling for that case?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll add some error handling.

Copy link
Copy Markdown
Member

@eschutho eschutho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just left one small comment/question.

@betodealmeida betodealmeida merged commit 2ce676d into apache:master Jul 27, 2021
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
* feat: call screenshot to store query_context

* Add unit test

* Move updateQueryContext to ExploreChartPanel

* Add error handling

* Fix code

* Fix logic
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
* feat: call screenshot to store query_context

* Add unit test

* Move updateQueryContext to ExploreChartPanel

* Add error handling

* Fix code

* Fix logic
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
* feat: call screenshot to store query_context

* Add unit test

* Move updateQueryContext to ExploreChartPanel

* Add error handling

* Fix code

* Fix logic
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
* feat: call screenshot to store query_context

* Add unit test

* Move updateQueryContext to ExploreChartPanel

* Add error handling

* Fix code

* Fix logic
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 First shipped in 1.3.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 1.3.0 First shipped in 1.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants