We are done,
Firstly checkout the pull request for the work product - https://github.com/scrapinghub/spidermon/pull/201
Project Repo - https://ift.tt/2zhmHI1
All tasks have been completed as per project proposal.
Cerberus validation library has now been integrated with Spidermon and its validation pipelines. Where users would be able to test their data items on custom schemas defined by them easily and with or no configuration.
It brings me great joy to end on a fulfilling note for contributing to Spidermon and the Scrapy Project as part of Google Summer of Code 2019, I am happy and content with the work produced.
The PR includes,
- CerberusValidator() class for item validation through Cerberus.
- Translator for translating errors for a better, unified system working with other validation methods.
- Complete integration with Scrapy pipelines, working with raw schema, URL's, and paths.
- Unit + integration tests for each component in place.
- Documentation for Cerberus Validation method.
For system testing, one could go ahead and use the pre-configured Quotes spider https://github.com/vipulgupta2048/testing_quotes and installing Spidermon from the master branch of my fork.
This project has been completed with long nights of reading and writing the code, learning new concepts on the fly and asking hundreds of pop-questions on Slack, that were answered duly by my mentors @ejulio @rennerocha as without their constant help, motivation, and guidance completing this uphill task wouldn't be ever possible.
Thank you all for reading,
You can check out more blogs here - https://mixstersite.wordpress.com/gsoc/
from Planet Python
via read more
No comments:
Post a Comment