Automated Testing: Add GitHub Actions workflow for Storybook smoke testing #69679
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Hi @mirka, When you have a moment I'd like to hear your thoughts on this Enhancement on our CI/CD Automated testing. |
There was a problem hiding this comment.
Yes, I think the overall approach of using test-storybook for automated smoke tests is solid!
I don't have the bandwidth to go super deep on reviewing/testing the workflow itself, but a few thoughts I had:
- Any reason we're doing the
pidthing rather than something more like the official example here? - And if we do switch to the
build-based approach like the example above, maybe we can save time by running the smoke tests in this existing workflow?
|
Thank you @mirka for your feedback. I have implemented the necessary changes in this commit: 06a6c1c. This is the related CI test action from the above commit: Storybook build and Smoke Tests I've now integrated the concurrently package to properly handle process termination when tests finish. The smoke tests have been incorporated into the existing Since you mentioned you have limited bandwidth, I'll tag @Mamaduka and @t-hamano to continue the review process from here. Thank you for your guidance! |
| - name: Install Playwright dependencies | ||
| run: npx playwright install | ||
|
|
||
| - name: Serve Storybook and run tests |
There was a problem hiding this comment.
Can "serve" and "run tests" parts be split into separate steps? It can make it more clear what is actually failing when issues are encountered.
There was a problem hiding this comment.
Ok, so all the Playwright tests we added here only seem to be adding a couple minutes to the previous version of the Storybook workflow, which seems reasonable!
Let's keep an eye out for anomalies, and tweak or remove if this turns out to be more trouble than it's worth 😂 I'll preemptively suggest a possible optimization: Historically, certain sections of the Storybook have been more prone to breakage (e.g. Block Editor Playground), so if testing every story ends up being too costly, we could maybe limit it to the problem sections.
Thanks @im3dabasia for your persistence here!
|
Hey @mirka , Could we merge these and see how it performs? We'd get a fair idea on if any improvements are needed after testing it in the Pipeline. Let me know, if further changes are needed . Regards. |
|
All of the feedback has been addressed, and the GitHub Action logs seem to be as expected, so I've merged it. Let's follow up and improve it as needed. |
…sting (WordPress#69679) * feat: Add test-runner and setup smoke tests for storybook * feat: Add build command * fix: Script command in package.json * restore: package-lock from trunk * restore: Old package.json * feat: Add storybook package * fix: Add node in types * remove: test-runner dep * fix: Update the smoke test run command * feat: Dev server to work in bg mode, sleep logic * fix: yaml file run command * chore: Add playwright install command * fix: Add concurrent exition logic when tests are run * fix: Increase stories timeout and added sleep * fix: Add timeout between jobs * chore: Add the smoke tests to storybook-check workflow * chore: Remove the storybook-test yaml * fix: Use http-server and removed sleep time * fix: Update the static path in the yaml * chore: Add with deps flag in playwright install Co-authored-by: im3dabasia <im3dabasia1@git.wordpress.org> Co-authored-by: mirka <0mirka00@git.wordpress.org> Co-authored-by: desrosj <desrosj@git.wordpress.org>
What?
Closes #41456
Why?
Code changes can break Storybook stories without anyone noticing until much later. Since our Storybook contains realistic usage examples, broken stories may indicate issues for third-party consumers.
How?
Added a GitHub Actions workflow that:
This catches breaking changes early before they're merged.
Screenshots
This is the Action Link
