Monitoring traffic of your Github repositories using Python and Google Cloud Platform — Part 1
Photo by Paweł Czerwiński on Unsplash
It is an article about monitoring your Github open-source repositories traffic. Unfortunately, you can see these statistics only by accessing each repository step by step. You may not want to access them at all… But if you do, you can use this small tool.
Technical stack:
And from the perspective of $ — this solution is zero cost because of the free quota you have in Google Cloud Platform. No ads — I just like using free opportunities.
So, the main concept is to get top referrers from Github for each your public repository and then store this data in firestore based on the date to be able to create a report on the weekly basis (in the next part).
In this part, we are going only to get top referrers from Github. The main and only code:
Cloud Function to get top referrers for each of your open-source Github repositories.
requirements.txt file:
PyGithub==1.43.8
google-cloud-firestore==1.4.0
You are going to need a Github personal access token to be able to make requests to Github API to get all your public repositories and then get traffic data from them. You can get it here.
Github personal access token page.
Click Generate new token.
Github new personal access token page. Click only `public_repo` scope.
Click Generate token.
From the GCP perspective — you need to have a created project, enabled Cloud Functions API and created a Cloud Firestore database and that’s all.
Docs to deploy Cloud Function in several ways can be found here.
I am using a gcloud tool from my local machine.
gcloud config set core/project <your-project-name>
gcloud functions deploy parse_github_repos_traffic --runtime python37 --trigger-http --set-env-vars=GITHUB_TOKEN=<your-github-token>
Newly created Cloud function.
To test the function, click on it, then go to the Testing tab.
A testing tab of your Cloud Function.
And press Test the function. And you should be able to see something like this.
A testing tab of your Cloud Function after pressing a Test the function button.
And this data is also available in Cloud Firestore (for future analytics).
Cloud Firestore with data.
That is all for this part. In the next part we are going to set up a scheduler to run call this function weekly and analyze the data we got to create a report.
Thanks for the attention to the topic, feel free to leave your questions in the comments for discussion.
Monitoring traffic of your Github repositories using Python and Google Cloud Platform — Part 1 was originally published in python4you on Medium, where people are continuing the conversation by highlighting and responding to this story.
from Planet Python
via read more
No comments:
Post a Comment