Brett Cannon: What the heck is pyproject.toml?

Recently on Twitter there was a maintainer of a Python project who had a couple of bugs filed against their project due to builds failing (this particular project doesn't provide wheels, only sdists). Eventually it came out that the project was using a pyproject.toml file because that's how you configure Black and not for any other purpose. This isn't the first time I have seen setuptools users use pyproject.toml because they were "told to by <insert name of tool>" without knowing the entire point behind the file. And so I decided to write this blog post to try and explain to setuptools users why pyproject.toml exists and what it does as it's the future of packaging in the Python ecosystem (if you are not a conda user 😉).

PEP 518 and pyproject.toml

I've blogged about this before, but the purpose of PEP 518 was to come up with a way for projects to specify what build tools they required. That's it, real simple and straightforward. Before PEP 518 and the introduction of pyproject.toml there was no way for a project to tell a tool like pip what build tools it required in order to build a wheel (let alone an sdist). Now setuptools has a setup_requires argument to specify what is necessary to build a project, but your can't read that setting unless you have setuptools installed, which meant you couldn't declare you needed setuptools to read the setting in setuptools. This chicken-and-egg problem is why tools like virtualenv install setuptools by default and why pip always injects setuptools and wheel when running a setup.py file regardless of whether you explicitly installed  it. Oh, and don't even try to rely on a specific version of setuptools for buildling your project as there was no way to specify that; you had to make do with whatever the user happened to have installed.

But PEP 518 and pyproject.toml changed that. Now a tool like pip can read pyproject.toml, see what build tools are specified in it, and install those in a virtual environment to build your project. That means you can rely on a specific version of setuptools and 'wheel' if you want. Heck, you can even build with a tool other than setuptools if you want (e.g. flit or Poetry, but since these other tools require pyproject.toml their users are already familiar with what's going on). The key point is assumptions no longer need to be made about what is necessary to build your project, which frees up the packaging ecosystem to experiment and grow.

PEP 517 and building wheels

With PEP 518 in place, tools knew what needed to be available in order to build a project into a wheel (or sdist). But how do you produce a wheel or sdist from a project that has a pyproject.toml? This is where PEP 517 comes in. That PEP specifies how build tools are to be executed to build both sdists and wheels. So PEP 518 gets the build tools installed and PEP 517 gets them executed. This opens the door to using other tools by standardizing how to run build tools. Before, there was no standardized way to build a wheel or sdist except with python setup.py sdist bdist_wheel which isn't really flexible; there's no way for the tool running the build to pass in environment details as appropriate, for instance. PEP 517 helped solve that problem.

One other change that PEP 517 & 518 has led to is build isolation. Now that projects can specify arbitrary build tools, tools like pip have to build projects in virtual environments to make sure each project's build tools don't conflict with another project's build tool needs. This also helps with reproducible builds by making sure your build tools are consistent.

Unfortunately this frustrates some setuptools users when they didn't realize a setup.py files and/or build environment have become structured in such a way that they can't be built in isolation. For instance, one user was doing their builds offline and didn't have setuptools and 'wheel' cached in their wheelhouse, so when pip tried to build a project in isolation it failed as pip couldn't find setuptools and 'wheel' to install into the build virtual environment.

Tools standardizing on pyproject.toml

An interesting side-effect of PEP 518 trying to introduce a standard file that all projects should (eventually) have is that non-build development tools realized they now had a file where they could put their own configuration. I say this is interesting because originally PEP 518 disallowed this, but people chose to ignore this part of the PEP 😄.  We eventually updated the PEP to allow for this use-case since it became obvious people liked the idea of centralizing configuration data in a single file.

And so now projects like Black, coverage.py, towncrier, and tox (in a way) allow you to specify their configurations in pyproject.toml instead of in a separate file. Occasionally you do here people lament the fact that they are adding yet another configuration file to their project due to pyproject.toml. What I don't think people realize, though, is these project could have also created their own configuration files (and in fact both coverage.py and tox do support their own files). And so, thanks to projects consolidating around pyproject.toml, there's actually an argument to be made there are fewer configuration files than before thanks to pyproject.toml.

How to use pyproject.toml with setuptools

Hopefully I have convinced you to introduce pyproject.toml into your setuptools-based project so you get benefits like build isolation and the ability to specify the version of setuptools you want to depend on. Now you might be wondering what your pyproject.toml should consist of? Unfortunately no one has had the time to document all of this for setuptools, but luckily the issue tracking adding that document outlines what is necessary:

[build-system]
requires = ["setuptools >= 40.6.0", "wheel"]
build-backend = "setuptools.build_meta"

A pyproject.toml file for setuptools users

With that you get to participate in thePEP 517 world of standards! 😉 And as I said, you can now rely on a specific version of setuptools and get build isolation as well (which is why the current directory is not put on sys.path automatically; you will need sys.path.insert(0, os.dirname(__file__)) or equivalent if you're importing local files).

But there's a bonus if you use a pyproject.toml file with a setup.cfg configuration for setuptools: you don't need a setup.py file anymore! Since tools like pip are going to call setuptools using the PEP 517 API instead of setup.py it means you can delete that setup.py file!

Unfortunately there is one hitch with dropping the setup.py file: if you want editable installs you still need a setup.py shim, but that's true of any build tool that isn't setuptools as there isn't a standard for editable installs (yet; people have talked about standardizing it and sketched it out, but no one has had the time to implement a proof-of-concept and then the eventual PEP). Luckily the shim to keep editable installs is really small:

#!/usr/bin/env python

import setuptools

if __name__ == "__main__":
    setuptools.setup()

A setup.py shim for use with pyproject.toml and setup.cfg

You could even simplify this down to import setuptools; setuptools.setup() if you really wanted to.

Where all of this is going

What all of this comes down to is the Python packaging ecosystem is working towards basing itself on standards. And those standards are all working towards standardizing artifacts and how to work with them. For instance, if we all know how wheels are formatted and how to install them then you don't have to care about how the wheel is made, just that a wheel exists for the thing you want to install and that it follows the appropriate standards. If you keep pushing this out and standardize more and more it makes it much easier for tools to communicate via artifacts and provide freedom for people to use whatever software they want to produce those artifacts.

For instance, you may have noticed I keep saying "tools like pip" instead of just saying "pip". That's been entirely on purpose. By making all of these standards it means tools don't have to rely solely on pip to do things because "that's how pip does it". As an example, tox could install a wheel by itself by using a library like pep517 to do the building of a wheel and then use another library like distlib to do the wheel installation.

Standards also take out the guessing as to whether something is on purpose or not. This becomes important to make sure everyone agrees on how things should work. There's also coherency as standards start to build on each other and flow into one another nicely. There's also less arguing (eventually 😉) as everyone works toward the same thing that everyone agreed to earlier.

It also takes pressure off of setuptools. It doesn't have to try and be everything to everyone as people can now choose the tool that best fits their project and development style. Same goes for pip.

Besides, don't we all want the platypus? 😉

from Planet Python
via read more

Zero-with-Dot (Oleg Żero): Hidden Markov Model - A story of the morning insanity

Introduction

In this article, we present an example of an (im-)practical application of the Hidden Markov Model (HMM). It is an artifially constructed problem, where we create a case for a model, rather than applying a model to a particular case… although, maybe a bit of both.

Here, we will rely on the code we developed earlier , and discussed in the earlier article: “Hidden Markov Model - Implementation from scratch”, including the mathematical notation. Feel free to take a look. The story we are about to tell contains modeling of the problem, uncovering the hidden sequence and training of the model.

Let the story begin…

Picture the following scenario: It’s at 7 a.m. You’re preparing to go to work. In practice, it means that you are running like crazy between different rooms. You spend some random amount of time in each, doing something, hoping to get everything you need to be sorted before you leave.

Sounds familiar?

To make things worse, your girlfriend (or boyfriend) has cats. The little furball wants to eat. Due to the morning hustle, it is uncertain whether you would remember to feed it. If you don’t, the cats will be upset… and so will your girlfriend if she finds out.

Modeling the situation

Say your flat has four rooms. That is to include the kitchen, bathroom, living room and bedroom. You spend some random amount of time in each, and transition between the rooms with a certain probability. At the same time, where ever you go, you are likely to make some distinct kinds of noises. Your girlfriend hears these noises and, despite being still asleep, she can infer in which room you are spending your time.

And so she does that day by day. She wants to make sure that you do feed the cats.

However, since she can’t be there, all she can do is to place the cat food bag in a room where you supposedly stay the longest. Hopefully, that will increase the chances that you do feed the “beast” (and save your evening).

Markovian view

From the Markovian perspective there are rooms are the hidden states ( in our case). Every minute (or any other time constant), we transition from one room to another . The probabilities associated with the transitioning are the elements of matrix .

At the same time, there exist distinct observable noises your girfriend can hear:

• flushing toilet (most likely: bathroom),
• toothbrushing sound (most likely: bathroom),
• coffee machine (most likely: kitchen),
• opening the fridge (most likely: kitchen),
• TV commercials (most likely: living room),
• music on a radio (most likely: kitchen),
• washing dishes (most likely: kitchen),
• taking a shower (most likely bathroom,
• opening/closing a wardrobe (most likely: bedroom),
• ackward silence… (can be anywhere…).

The probabilities of their occurrence given a state is given by coefficients of the matrix . In principle, any of these could originate from you being in an arbitrary room (state). In practice, however, there is physically a little chance you pulled the toilet-trigger while being in the kitchen, thus some ’s will be close to zero.

Most importantly, as you hop from one room to the other, it reasonable to assume that whichever room you go to depends only on the room that you have just been to. In other words, the state at time depends on the state at time only, especially if your are half brain-dead with an attention span of a gold fish…

Uncovering the hidden states

The goal

For the first attempt, let’s assume that the probability coefficients are known. This means that we have a model , and our task is to estimate the latent sequence given the observation sequence, which corresponds to finding . In other words, the girlfriend wants to establish in what room do we spend the most time, given what she hears.

Initialization

Let’s initialize our and .

	bathroom	bedroom	kitchen	living room
bathroom	0.90	0.08	0.01	0.01
bedroom	0.01	0.90	0.05	0.04
kitchen	0.03	0.02	0.85	0.10
living room	0.05	0.02	0.23	0.70

	coffee	dishes	flushing	radio	shower	silence	television	toothbrush	wardrobe
bathroom	0.01	0.01	0.20	0.01	0.30	0.05	0.01	0.40	0.01
bedroom	0.01	0.01	0.01	0.10	0.01	0.30	0.05	0.01	0.50
kitchen	0.30	0.20	0.01	0.10	0.01	0.30	0.05	0.02	0.01
living room	0.03	0.01	0.01	0.19	0.01	0.39	0.39	0.01	0.03

bathroom	bedroom	kitchen	living room
0	1	0	0

1 2 3

... # all initializations as explained in the other article hml = HiddenMarkovLayer(A, B, pi) hmm = HiddenMarkovModel(hml)

Simulation

Having defined and , let’s see how a typical “morning insanity” might look like. Here, we assume that the whole “circus” lasts 30 minutes, with one-minute granularity.

1 2	observations, latent_states = hml.run(30) pd.DataFrame({'noise': observations, 'room': latent_states})

t	noise	room
0	radio	bedroom
1	wardrobe	bedroom
2	silence	bedroom
3	wardrobe	bedroom
4	silence	living room
5	coffee	bedroom
6	wardrobe	bedroom
7	wardrobe	bedroom
8	radio	bedroom
9	wardrobe	kitchen
…	…	…

The table above shows the first ten minutes of the sequence. We can see that it kind of makes sense, although we have to note that the girlfriend does not know what room we visited. This sequence is hidden from her.

However, as it is presented in the last article, we can guess what would be the statistically most favorable sequence of the rooms given the observations. The problem is addressed with the .uncover method.

1 2	estimated_states = hml.uncover(observations) pd.DataFrame({'estimated': estimated_states, 'real': latent_states})

t	estimated	real
0	bedroom	bedroom
1	bedroom	bedroom
2	bedroom	bedroom
3	bedroom	bedroom
4	bedroom	living room
5	bedroom	bedroom
6	bedroom	bedroom
7	bedroom	bedroom
8	bedroom	bedroom
9	bedroom	kitchen
…	…	…

Comparing the results, we get the following count:

	estimated time proportion	real time proportion
bathroom	1	2
bedroom	18	17
kitchen	4	10
living room	8	2

The resulting estimate gives 12 correct matches. Although this may seem like not much (only ~40% accuracy), it is 1.6 times better than a random guess.

Furthermore, we are not interested in matching the elements of the sequences here anyway. What interests us more is to find the room that you spend the most amount of time in. According to the simulation, you spend as much as 17 minutes in the bedroom. This estimate is off by one minute from the real sequence, which translates to ~6% relative error. Not that bad.

According to these results, the cat food station should be placed in the bedroom.

Training the model

In the last section, we have, relied on an assumption that the intrinsic probabilities of transition and observation are known. In other words, your girlfriend must have been watching your pretty closely, essentially collecting data about you. Otherwise, how else would she be able for formulate a model?

Although this may sound like total insanity, the good news is that our model is also trainable. Given a sequence of observations, it is possible to train the model and then use it to examine the hidden variables.

Let’s take an example sequence of what your girlfriend could have heard some crazy Monday morning. You woke up. Being completely silent for about 3 minutes, you went about to look for your socks in a wardrobe. Having found what you needed (or not), you went silent again for five minutes and flushed the toilet. Immediately after, you proceeded to take a shower (5 minutes), followed by brushing your teeth (3 minutes), although you turn the radio on in between. Once you were done, you turned the coffee machine on, watched TV (3 minutes), and did the dishes.

So, the observable sequence goes as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13

what_she_heard = ['silence']*3 \ + ['wardrobe'] \ + ['silence']*5 \ + ['flushing'] \ + ['shower']*5 \ + ['radio']*2 \ + ['toothbrush']*3 \ + ['coffee'] \ + ['television']*3 \ + ['dishes'] rooms = ['bathroom', 'bedroom', 'kitchen', 'living room'] pi = PV({'bathroom': 0, 'bedroom': 1, 'kitchen': 0, 'living room': 0})

The starting point is the bedroom, but and are unknown. Let’s initialize the model, and train it on the observation sequence.

1 2 3 4 5 6 7 8 9 10 11 12 13 14

np.random.seed(3) model = HiddenMarkovModel.initialize(rooms, list(set(what_she_heard))) model.layer.pi = pi model.train(what_she_heard, epochs=100) fig, ax = plt.subplots(1, 1, figsize=(10, 5)) ax.semilogy(model.score_history) ax.set_xlabel('Epoch') ax.set_ylabel('Score') ax.set_title('Training history') plt.grid() plt.show()

Figure 1. Training history using 100 epochs. To train means to maximise the score.

Figure 1.

Training history using 100 epochs. To train means to maximise the score.

Now, after training of the model, the prediction sequence goes as follows:

1 2 3 4

pd.DataFrame(zip( what_she_heard, model.layer.uncover(what_she_heard)), columns=['the sounds you make', 'her guess on where you are'])

t	the sound you make	her guess on where you are
0	silence	bedroom
1	silence	bedroom
2	silence	bedroom
3	wardrobe	bedroom
4	silence	bedroom
5	silence	bedroom
6	silence	bedroom
7	silence	bedroom
8	silence	bedroom
9	flushing	bathroom
10	shower	bathroom
11	shower	bathroom
12	shower	bathroom
13	shower	bathroom
14	shower	bathroom
15	radio	kitchen
16	radio	kitchen
17	toothbrush	living room
18	toothbrush	living room
19	toothbrush	living room
20	coffee	living room
21	television	living room
22	television	living room
23	television	living room
24	dishes	living room

state (guessed)	total time steps
bathroom	6
bedroom	9
kitchen	2
living room	8

According to the table above, it is evident that the cat food should be placed in the bedroom.

However, it is important to note that this result is somewhat a nice coincidence because the model was initialized from a purely random state. Consequently, we had no control over the direction it would evolve in the context of the labels. In other words, the naming for the hidden states is are simply abstract to the model. They are our convention, not the model’s. Consequently, the model could have just as well associated “shower” with the “kitchen” and “coffee” with “bathroom”, in which case the model would still be correct, but to interpret the results we would need to swap the labels.

Still, in our case, the model seems to have trained to output something fairly reasonable and without the need to swap the names.

Conclusion

Hopefully, we have shed a bit of light into this whole story of morning insanity using the Hidden Markov model approach.

In this short story, we have covered two study cases. The first case assumed that the probability coefficients were known. Using these coefficients, we could define the model and uncover the latent state sequence given the observation sequence. The second case represented the opposite situation. The probabilities were not known and so the model had to be trained first in order to output the hidden sequence.

Closing remark

The situation described here is a real situation that the author faces every day. And yes… the cats survived. ;)

from Planet Python
via read more

Stack Abuse: Reading and Writing MS Word Files in Python via Python-Docx Module

The MS Word utility from Microsoft Office suite is one of the most commonly used tools for writing text documents, both simple and complex. Though humans can easily read and write MS Word documents, assuming you have the Office software installed, often times you need to read text from Word documents within another application.

For instance, if you are developing a natural language processing application in Python that takes MS Word files as input, you will need to read MS Word files in Python before you can process the text. Similarly, often times you need to write text to MS Word documents as output, which could be a dynamically generated report to download, for example.

In this article, article you will see how to read and write MS Word files in Python.

Installing Python-Docx Library

Several libraries exist that can be used to read and write MS Word files in Python. However, we will be using the python-docx module owing to its ease-of-use. Execute the following pip command in your terminal to download the python-docx module as shown below:

$ pip install python-docx

Reading MS Word Files with Python-Docx Module

In this section, you will see how to read text from MS Word files via the python-docx module.

Create a new MS Word file and rename it as "my_word_file.docx". I saved the file in the root of my "E" directory, although you can save the file anywhere you want. The my_word_file.docx file should have the following content:

To read the above file, first import the docx module and then create an object of the Document class from the docx module. Pass the path of the my_word_file.docx to the constructor of the Document class, as shown in the following script:

import docx

doc = docx.Document("E:/my_word_file.docx")

The Document class object doc can now be used to read the content of the my_word_file.docx.

Reading Paragraphs

Once you create an object of the Document class using the file path, you can access all the paragraphs in the document via the paragraphs attribute. An empty line is also read as a paragraph by the Document. Let's fetch all the paragraphs from the my_word_file.docx and then display the total number of paragraphs in the document:

all_paras = doc.paragraphs
len(all_paras)

Output:

10

Now we'll iteratively print all the paragraphs in the my_word_file.docx file:

for para in all_paras:
    print(para.text)
    print("-------")

Output:

-------
Introduction
-------

-------
Welcome to stackabuse.com
-------
The best site for learning Python and Other Programming Languages
-------
Learn to program and write code in the most efficient manner
-------

-------
Details
-------

-------
This website contains useful programming articles for Java, Python, Spring etc.
-------

The output shows all of the paragraphs in the Word file.

We can even access a specific paragraph by indexing the paragraphs property like an array. Let's print the 5th paragraph in the file:

single_para = doc.paragraphs[4]
print(single_para.text)

Output:

The best site for learning Python and Other Programming Languages

Reading Runs

A run in a word document is a continuous sequence of words having similar properties, such as similar font sizes, font shapes, and font styles. For example, if you look at the second line of the my_word_file.docx, it contains the text "Welcome to stackabuse.com", here the text "Welcome to" is in plain font, while the text "stackabuse.com" is in bold face. Hence, the text "Welcome to" is considered as one run, while the bold faced text "stackabuse.com" is considered as another run.

Similarly, "Learn to program and write code in the" and "most efficient manner" are treated as two different runs in the paragraph "Learn to program and write code in the most efficient manner".

To get all the runs in a paragraph, you can use the run property of the paragraph attribute of the doc object.

Let's read all the runs from paragraph number 5 (4th index) in our text:

single_para = doc.paragraphs[4]
for run in single_para.runs:
    print(run.text)

Output:

The best site for
learning Python
 and Other
Programming Languages

In the same way, the following script prints all the runs from the 6th paragraph of the my_word_file.docx file:

second_para = doc.paragraphs[5]
for run in second_para.runs:
    print(run.text)

Output:

Learn to program and write code in the
most efficient manner

Writing MS Word Files with Python-Docx Module

In the previous section, you saw how to read MS Word files in Python using the python-docx module. In this section, you will see how to write MS Word files via the python-docx module.

To write MS Word files, you have to create an object of the Document class with an empty constructor, or without passing a file name.

mydoc = docx.Document()

Writing Paragraphs

To write paragraphs, you can use the add_paragraph() method of the Document class object. Once you have added a paragraph, you will need to call the save() method on the Document class object. The path of the file to which you want to write your paragraph is passed as a parameter to the save() method. If the file doesn't already exist, a new file will be created, otherwise the paragraph will be appended at the end of the existing MS Word file.

The following script writes a simple paragraph to a newly created MS Word file named "my_written_file.docx".

mydoc.add_paragraph("This is first paragraph of a MS Word file.")
mydoc.save("E:/my_written_file.docx")

Once you execute the above script, you should see a new file "my_written_file.docx" in the directory that you specified in the save() method. Inside the file, you should see one paragraph which reads "This is first paragraph of a MS Word file."

Let's add another paragraph to the my_written_file.docx:

mydoc.add_paragraph("This is the second paragraph of a MS Word file.")
mydoc.save("E:/my_written_file.docx")

This second paragraph will be appended at the end of the existing content in my_written_file.docx.

Writing Runs

You can also write runs using the python-docx module. To write runs, you first have to create a handle for the paragraph to which you want to add your run. Take a look at the following example to see how it's done:

third_para = mydoc.add_paragraph("This is the third paragraph.")
third_para.add_run(" this is a section at the end of third paragraph")
mydoc.save("E:/my_written_file.docx")

In the script above we write a paragraph using the add_paragraph() method of the Document class object mydoc. The add_paragraph() method returns a handle for the newly added paragraph. To add a run to the new paragraph, you need to call the add_run() method on the paragraph handle. The text for the run is passed in the form of a string to the add_run() method. Finally, you need to call the save() method to create the actual file.

Writing Headers

You can also add headers to MS Word files. To do so, you need to call the add_heading() method. The first parameter to the add_heading() method is the text string for header, and the second parameter is the header size. The header sizes start from 0, with 0 being the top level header.

The following script adds three headers of level 0, 1, and 2 to the file my_written_file.docx:

mydoc.add_heading("This is level 1 heading", 0)
mydoc.add_heading("This is level 2 heading", 1)
mydoc.add_heading("This is level 3 heading", 2)
mydoc.save("E:/my_written_file.docx")

Adding Images

To add images to MS Word files, you can use the add_picture() method. The path to the image is passed as a parameter to the add_picture() method. You can also specify the width and height of the image using the docx.shared.Inches() attribute. The following script adds an image from the local file system to the my_written_file.docx Word file. The width and height of the image will be 5 and 7 inches, respectively:

mydoc.add_picture("E:/eiffel-tower.jpg", width=docx.shared.Inches(5), height=docx.shared.Inches(7))
mydoc.save("E:/my_written_file.docx")

After executing all the scripts in the Writing MS Word Files with Python-Docx Module section of this article, your final my_written_file.docx file should look like this:

In the output, you can see the three paragraphs that you added to the MS word file, along with the three headers and one image.

Conclusion

The article gave a brief overview of how to read and write MS Word files using the python-docx module. The article covers how to read paragraphs and runs from within a MS Word file. Finally, the process of writing MS Word files, adding a paragraph, runs, headers, and images to MS Word files have been explained in this article.

from Planet Python
via read more

Interview with a Head of AI: Vladimir Rybakov

Interview with a Head of AI: Vladimir Rybakov Some time ago we sat down with Vladimir Rybakov, Head of Data Science at WaveAccess and a master communicator of business to ML teams and ML problems to business people. We talked about: How to solve problems fast under extreme pressure How taking responsibility can help you […]

The post Interview with a Head of AI: Vladimir Rybakov appeared first on neptune.ai.

from Planet SciPy
read more

PyCoder’s Weekly: Issue #414 (March 31, 2020)

#414 – MARCH 31, 2020
View in Browser »

---

Automatically Finding Codenames Clues With GloVe Vectors

In the Czech boardgame Codenames, one player must come up with a single-word clue that prompts their teammates to select certain words from a 5x5 board while simultaneously avoiding a “bomb” word that, if selected, causes the team to lose. In this article, James Somers explores how to generate clues automatically using Global Vectors for Word Representations—with surprising results.
JAMES SOMERS

Learn Python Skills While Creating Games

In this episode, Christopher interviews Jon Fincher from the Real Python Team. Jon talks about his recent articles on PyGame and Arcade. They discuss if game programming is a good way to develop your Python programming skills, and if a game would make a good portfolio piece.
REAL PYTHON podcast

Automate & Standardize Code Reviews for Python

Take the hassle out of code reviews - Codacy flags errors automatically, directly from your Git workflow. Customize standards on coverage, duplication, complexity & style violations. Use in the cloud or on your servers for 30 different languages. Get started for free →
CODACY sponsor

SimPy: Simulating Real-World Processes With Python

See how you can use the SimPy package to model real-world processes with a high potential for congestion. You’ll create an algorithm to approximate a complex system, and then you’ll design and run a simulation of that system in Python.
REAL PYTHON

psycopg3: A First Report

“What’s left? Well, a lot! Now that the basic machinery is in place, and Python can send and retrieve bytes to and from Postgres, it’s time to attack the adaptation layer.”
DANIELE VARRAZZO

Cognitive Biases in Software Development

“In this post, I will try to answer the question: why do we feel weird about technical solutions?” Not strictly Python-focused, but absolutely worth a read.
STANISLAV MYACHENKOV

Learning Pandas by Exploring COVID-19 Data

“Use the pandas data analysis tool to explore the free COVID-19 data set provided by the European Centre for Disease Prevention and Control.”
MATT MAKAI

How Long Did It Take You to Learn Python?

“Wait, don’t answer that. It doesn’t matter.”
NED BATCHELDER

Discussions

Are F-Strings Good Practice?

REDDIT

What Resources Would You Recommend to Learn Socket Programming?

LOBSTE.RS

Python Jobs

Python Tutorial Authors Wanted (100% Remote)

Real Python

Software Engineer - PyTorch (Remote)

CyberCoders

Senior Python Engineer (Remote)

CyberCoders

Python Quant Developer With Data Focus (Remote)

eFinancial Careers

Senior Python Developer With Django or Flask (Remote)

Botsford Associates LLC

More Python Jobs >>>

Articles & Tutorials

Django vs. Flask in 2019: Which Framework to Choose [2019]

”[…] Django and Flask are by far the two most popular Python web frameworks… In the end, both frameworks are used to develop web applications. The key difference lies in how they achieve this goal. Think of Django as a car and Flask as a bike. Both can get you from point A to point B, but their approaches are quite different.”
MICHAEL HERMAN

How to Calculate Feature Importance With Python

“Feature importance scores play an important role in a predictive modeling project, including providing insight into the data, insight into the model, and the basis for dimensionality reduction and feature selection that can improve the efficiency and effectiveness of a predictive model on the problem. In this tutorial, you will discover feature importance scores for machine learning in Python.”
JASON BROWNLEE

Get Inspiration for Your Next Machine Learning Project

Python is a leading language for machine learning, and businesses worldwide are using the technology to get ahead. Explore 22 examples of how ML is being applied across four industries →
STXNEXT sponsor

Using Markdown in Django

“As developers, we rely on static analysis tools to check, lint and transform our code. We use these tools to help us be more productive and produce better code. However, when we write content using markdown the tools at our disposal are scarce. In this article we describe how we developed a Markdown extension to address challenges in managing content using Markdown in Django sites.”
HAKI BENITA

Using WSL to Build a Python Development Environment on Windows

Learn how to set-up a development environment on Windows that leverages the power of the Windows Subsystem for Linux (WSL). From installing WSL, interfacing with Windows Terminal, setting up VS Code, and even running graphical Linux apps, this comprehensive tutorial covers everything you need to know to set-up a development workflow.
CHRIS MOFFITT

How to Use any() in Python

If you’ve ever wondered how to simplify complex conditionals by determining if at least one in a series of conditions is true, then look no further. This tutorial will teach you all about how to use any() in Python to do just that.
REAL PYTHON

Profile, Understand & Optimize Code Performance

You can’t improve what you can’t measure. Profile and understand Python code’s behavior and performance (Wall-time, I/O, CPU, HTTP requests, SQL queries). Browse through appealing graphs. Blackfire.io is now available as Public Beta. New features added regularly.
BLACKFIRE sponsor

Comparing Python Objects the Right Way: “is” vs “==”

Learn when to use the Python is, is not, == and != operators. You’ll see what these comparison operators do under the hood, dive into some quirks of object identity and interning, and define a custom class.
REAL PYTHON video

The Usefulness of Python’s Permutations and Combinations Functions

Learn how the permutations() and combinations() functions in Python’s itertools module can help you writer cleaner and faster for loops.
KEVIN DAWE

TLDR Newsletter: Byte Sized News for Techies

TLDR is a daily, curated newsletter with links and TLDRs of the most interesting stories in tech, science, and programming.
TLDRNEWSLETTER.COM

Projects & Code

streamz: Build Pipelines to Manage Continuous Streams of Data

GITHUB.COM/PYTHON-STREAMZ

pyliveupdate: Runtime Code Manipulation Framework for Profiling, Debugging and Bugfixing

GITHUB.COM/DEVOPSPP

meshy: Interactive Graph of Your Django Model Structure

GITHUB.COM/MESHY • Shared by Charlie Denton

rain: Live Example to Illustrate Python Packaging, Testing, Building, and Deploying

GITHUB.COM/ANKUR-GUPTA

pm4py-source: Process Mining for Python

GITHUB.COM/PM4PY

convtools: Generates Python Code of Conversions, Aggregations, Joins

GITHUB.COM/ITECHART-ALMAKOV

rapidfuzz: Rapid Fuzzy String Matching in Python

GITHUB.COM/RHASSPY

Python 3 Maze Generator With Disjoint Set

GITHUB.COM/SCHEDUTRON

glow: Compiler for Neural Network Hardware Accelerators

GITHUB.COM/PYTORCH

pywharf: Host Your Python Package on GitHub

GITHUB.COM/PYWHARF • Shared by HUNT ZHAN

typer-cli: Run Typer Scripts With Auto-Completion

GITHUB.COM/TIANGOLO • Shared by Sebastián Ramírez

Events

PyLadies Remote Lightning Talks

March 28th, 2020 (Remote via Zoom)
PYLADIES

---

Happy Pythoning!
This was PyCoder’s Weekly Issue #414.
View in Browser »

---

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

from Planet Python
via read more

Continuum Analytics Blog: Securing Pangeo with Dask Gateway

This post is also available on the Pangeo blog. Over the past few weeks, we have made some exciting changes to Pangeo’s cloud deployments. These changes will make using Pangeo’s clusters easier for users while…

The post Securing Pangeo with Dask Gateway appeared first on Anaconda.

from Planet Python
via read more

Real Python: Comparing Python Objects the Right Way: "is" vs "=="

There’s a subtle difference between the Python identity operator (is) and the equality operator (==). Your code can run fine when you use the Python is operator to compare numbers, until it suddenly doesn’t. You might have heard somewhere that the Python is operator is faster than the == operator, or you may feel that it looks more Pythonic. However, it’s crucial to keep in mind that these operators don’t behave quite the same.

The == operator compares the value or equality of two objects, whereas the Python is operator checks whether two variables point to the same object in memory. In the vast majority of cases, this means you should use the equality operators == and !=, except when you’re comparing to None.

In this course, you’ll learn:

• What the difference is between object equality and identity
• When to use equality and identity operators to compare objects
• What these Python operators do under the hood
• Why using is and is not to compare values leads to unexpected behavior
• How to write a custom __eq__() class method to define equality operator behavior

---

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

from Planet Python
via read more

EuroPython: EuroPython 2020: Online conference from July 23-26

In the last two weeks, we have discussed and investigated concepts around running this year’s EuroPython conference as an online conference. We have looked at conference tools, your feedback, drafted up ideas on what we can do to make the event interesting and what we can accomplish given our limited resources.

Today, we are happy to announce that we will be running

EuroPython 2020
from July 23 - 26 2020
as an online conference

We are planning the following structure:

• 2 conference days with keynotes, talks, lightning talks, poster session
  (Thursday and Friday)

• 2 sprint days with multiple sprint teams
  (Saturday and Sunday)

Attending the conference days will require a ticket, participating in the sprint days will be free.

We will publish additional information on the new format as it becomes available. A few updates for today (more details will follow in separate blog posts):

Call for Papers (CFP)

With the new plan in place, we will extend and retarget the CFP we had been running in the last three weeks to the online setup.

Please note that we will not have training sessions at EuroPython 2020. We will have keynotes, 30 and 45-minute talks, panels, interactive sessions, as well as try to come up with a format for doing posters and lightning talks.

Unlike for our in-person event, speakers will get free tickets to the event, since we don’t have to cover catering costs.

We need your help

Given that we had already put a lot of work into the in-person event organization, a lot of which we’ll now have to adapt or redo for the online set up in the few months ahead of us, we will need more help from the community to make this happen.

If you would like to help, please write to board@europython.eu. We are specifically looking for people with experience using online conference tools to help host tracks during the conference days.

Sponsoring

As for the in-person event, we will have sponsorship packages available for the online event as well. Because the format is different, we’ll have to adjust the packages we had intended for the in-person to an online setup.

If you are interested in sponsoring EuroPython 2020, please write to sponsoring@europython.eu. We will then send you more details, as they become available.

Thanks,
–
EuroPython 2020 Team
https://ep2020.europython.eu/
https://www.europython-society.org/

from Planet Python
via read more

Python Software Foundation: PSF's Projected 2020 Financial Outcome

The Python Software Foundation (PSF) is a 501(c)(3) non-profit organization dedicated to the Python community and programming language, as well as running PyCon US. Since PyCon US 2020 was cancelled, the community has asked how the PSF’s finances will be affected. Let us take a look at the projected 2020 financial outcome.

Bottom Line

As of today, the PSF will use approximately $627,000 from our financial reserve:

	Expenses	Revenue
PSF	$1,300,000	$550,000	-$750,000
PyCon	$280,000	$403,000	$123,000
		Total	-$627,000

If you are interested in how we arrived at these estimates, continue reading to learn about our projected expenses and revenue for this year. 

Expenses

PyCon US 2020

Pittsburgh and its vendors have been incredibly helpful in reducing or eliminating most of the 2020 conference minimums and cancellation fees. We estimate $280,000 in expenses for pre-conference work related to website/logo design and nonrefundable deposits. In addition, we budgeted significant funds to support travel grantees with non-reimbursable costs, as well as executing PyCon 2020 remote content. Once travel grants and instructor fees are complete, we will revise the expense total. 

PSF

Through March 2020, the PSF awarded several grants*, expended legal fees to protect PyLadies trademarks in dozens of countries, and employed staff. The PSF is projected to spend $1,300,000 in 2020.

Revenue

PyCon US 2020

PyCon US registration and sponsorship revenue is used to produce PyCon, with the largest costs going to food, audio-visual services, and travel grants. 

Our staff works to create the best and most affordable attendee experience possible with the added benefit that 100% of net proceeds fund the PSF. For 2020, we estimated PyCon’s net income at $720,000. As of today, we are estimating PyCon's net income to be $123,000, thanks to individual donations and sponsorship fees. 

PSF

PSF 2020 sponsorships are estimated at $350,000. COVID-19 is impacting financial markets and job security, so we expect individual donations and memberships to decrease in 2020 by 55% from 2019 to around $200,000. 

How can you help?

The PSF’s financial reserve is crucial, as we experienced during the economic downturn of 2008 and again in 2020. The cash reserve prepares us for economic impacts, events out of our control, and provides a stable environment with health benefits for our employees, even during this difficult time. 

Here are ways community members can help and get involved:

• Become a free or supporting member of the PSF to get involved in our future
• Donate some or all of your PyCon registration (thank you to those that already have)
• Donate directly to the PSF
• Donate some or all of your company's PyCon's sponsorship (thank you to those that already have)
• Ask your employer to sponsor the PSF
• Ask your employer if they match donations to 501(c)(3) non-profits, and ask for your donations to the PSF to be matched
• Sign up for the PSF’s free newsletter to stay up to date

We wish our entire community good health.

* PSF Grants: When PyCon 2020 was cancelled, the PSF paused its Grants Program until we can find virtual options and other ways to support events, as well as fully understand the PSF’s financial situation.

from Planet Python
via read more

PSF's Projected 2020 Financial Outcome

The Python Software Foundation (PSF) is a 501(c)(3) non-profit organization dedicated to the Python community and programming language, as well as running PyCon US. Since PyCon US 2020 was cancelled, the community has asked how the PSF’s finances will be affected. Let us take a look at the projected 2020 financial outcome.

Bottom Line

As of today, the PSF will use approximately $627,000 from our financial reserve:

	Expenses	Revenue
PSF	$1,300,000	$550,000	-$750,000
PyCon	$280,000	$403,000	$123,000
		Total	-$627,000

If you are interested in how we arrived at these estimates, continue reading to learn about our projected expenses and revenue for this year. 

Expenses

PyCon US 2020

Pittsburgh and its vendors have been incredibly helpful in reducing or eliminating most of the 2020 conference minimums and cancellation fees. We estimate $280,000 in expenses for pre-conference work related to website/logo design and nonrefundable deposits. In addition, we budgeted significant funds to support travel grantees with non-reimbursable costs, as well as executing PyCon 2020 remote content. Once travel grants and instructor fees are complete, we will revise the expense total. 

PSF

Through March 2020, the PSF awarded several grants*, expended legal fees to protect PyLadies trademarks in dozens of countries, and employed staff. The PSF is projected to spend $1,300,000 in 2020.

Revenue

PyCon US 2020

PyCon US registration and sponsorship revenue is used to produce PyCon, with the largest costs going to food, audio-visual services, and travel grants. 

Our staff works to create the best and most affordable attendee experience possible with the added benefit that 100% of net proceeds fund the PSF. For 2020, we estimated PyCon’s net income at $720,000. As of today, we are estimating PyCon's net income to be $123,000, thanks to individual donations and sponsorship fees. 

PSF

PSF 2020 sponsorships are estimated at $350,000. COVID-19 is impacting financial markets and job security, so we expect individual donations and memberships to decrease in 2020 by 55% from 2019 to around $200,000. 

How can you help?

The PSF’s financial reserve is crucial, as we experienced during the economic downturn of 2008 and again in 2020. The cash reserve prepares us for economic impacts, events out of our control, and provides a stable environment with health benefits for our employees, even during this difficult time. 

Here are ways community members can help and get involved:

• Become a free or supporting member of the PSF to get involved in our future
• Donate some or all of your PyCon registration (thank you to those that already have)
• Donate directly to the PSF
• Donate some or all of your company's PyCon's sponsorship (thank you to those that already have)
• Ask your employer to sponsor the PSF
• Ask your employer if they match donations to 501(c)(3) non-profits, and ask for your donations to the PSF to be matched
• Sign up for the PSF’s free newsletter to stay up to date

We wish our entire community good health.

* PSF Grants: When PyCon 2020 was cancelled, the PSF paused its Grants Program until we can find virtual options and other ways to support events, as well as fully understand the PSF’s financial situation.

from Python Software Foundation News
via read more

Comparing Python Objects the Right Way: "is" vs "=="

There’s a subtle difference between the Python identity operator (is) and the equality operator (==). Your code can run fine when you use the Python is operator to compare numbers, until it suddenly doesn’t. You might have heard somewhere that the Python is operator is faster than the == operator, or you may feel that it looks more Pythonic. However, it’s crucial to keep in mind that these operators don’t behave quite the same.

The == operator compares the value or equality of two objects, whereas the Python is operator checks whether two variables point to the same object in memory. In the vast majority of cases, this means you should use the equality operators == and !=, except when you’re comparing to None.

In this course, you’ll learn:

• What the difference is between object equality and identity
• When to use equality and identity operators to compare objects
• What these Python operators do under the hood
• Why using is and is not to compare values leads to unexpected behavior
• How to write a custom __eq__() class method to define equality operator behavior

---

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

from Real Python
read more

Codementor: Michael Kennedy almost learned Python in the 90s... and other things I learned recording his DevJourney

Michael Kennedy is a podcaster and a trainer. After interviewing him for the DevJourney podcast, here are the key takeways I personally took out of the discussion.

from Planet Python
via read more

Kushal Das: Introducing ManualBox project

One of the major security features of the QubesOS is the file vaults, where access to specific files can only happen via user input in the GUI applet. Same goes to the split-ssh, where the user has to allow access to the ssh key (actually on a different VM).

I was hoping to have similar access control to important dotfiles with passwords, ssh private keys, and other similar files on my regular desktop system. I am introducing ManualBox which can provide similarly access control on normal Linux Desktops or even on Mac.

How to install?

Follow the installation guide on the Mac in the wiki. For Linux, we are yet to package the application, and you can directly run from the source (without installing).

git clone https://github.com/kushaldas/manualbox.git
cd manualbox

On Fedora

sudo dnf install python3-cryptography python3-qt5 python3-fusepy python3-psutil fuse -y

On Debian

sudo apt install python3-cryptography python3-pyqt5 python3-fusepy python3-psutil fuse

Usage guide

To start the application from source:

On Linux:

./devscripts/manualbox

On Mac:

Click on the App icon like any other application.

If you are running the tool for the first time, it will create a new manualbox and mount it in ~/secured directory, it will also give you the password, please store it somewhere securely, as you will need it to mount the filesystem from the next time.

After selecting (or you can directly type) the mount path (must be an empty directory), you should type in the password, and then click on the Mount button.

Now, if you try to access any file, the tool will show a system notification, and you can either Allow or Deny via the following dialog.

Every time you allow file access, it shows the notification message via the system tray icon.

To exit the application, first click on the Unmount, and right-click on the systray icon, and click on the Exit or close via window close button.

Usage examples (think about your important dotfiles with passwords/tokens)

Note: If you open the mounted directory path from a GUI file browser, you will get too many notifications, as these browsers will open the file many times separately. Better to have you GUI application/command line tool to use those files as required.

Thunderbird

You can store your thuderbird profile into this tool. That way, thunderbird needs your permission for access when you start the application.

ls -l ~/.thunderbird/
# now find your right profile (most people have only one)
mv ~/.thunderbird/xxxxxx.default/logins.json ~/secured/
ln -s ~/secured/logins.json ~/.thunderbird/xxxxxx.default/logins.json

SSH private key

mv ~/.ssh/id_rsa ~/secured/
ln -s ~/secured/id_rsa ~/.ssh/id_rsa

If you have any issues, please file issues or even better a PR along with the issue :)

from Planet Python
via read more

Programiz: Python main function

In this tutorial, we will learn how to use a Python program's __name__ attribute to run it dynamically in different contexts.

from Planet Python
via read more

Mike Driscoll: Python 101 – Learning About Dictionaries

Dictionaries are another fundamental data type in Python. A dictionary is a key, value pair. Some programming languages refer to them as hash tables. They are described as a mapping object that maps hashable values to arbitrary objects.

A dictionary’s keys must be immutable, that is, unable to change. Starting in Python 3.7, dictionaries are ordered. What that means is that when you add a new key, value pair to a dictionary, it remembers what order they were added. Prior to Python 3.7, this was not the case and you could not rely on insertion order.

You will learn how to do the following in this chapter:

• Create dictionaries
• Access dictionaries
• Dictionary methods
• Modifying dictionaries
• Deleting from your dictionary

Let’s start off by learning about creating dictionaries!

You can create a dictionary in a couple of different ways. The most common method is by placing a comma-separated list key: value pairs within curly braces.

Let’s look at an example:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict
{'email': 'jdoe@gmail.com', 'first_name': 'James', 'last_name': 'Doe'}

You can also use Python’s built-in dict() function to create a dictionary. dict() will accept a series of keyword arguments (i.e. 1=’one’, 2=’two’, etc), a list of tuples or another dictionary.

Here are a couple of examples:

>>> numbers = dict(one=1, two=2, three=3)
>>> numbers
{'one': 1, 'three': 3, 'two': 2}
>>> info_list = [('first_name', 'James'), ('last_name', 'Doe'), ('email', 'jdoes@gmail.com')]
>>> info_dict = dict(info_list)
>>> info_dict
{'email': 'jdoes@gmail.com', 'first_name': 'James', 'last_name': 'Doe'}

The first example uses dict() on a series of keyword arguments. You will learn more about these when you learn about functions. You can think of keyword arguments as a series of keywords with the equals sign between them and their value.

The second example shows you how to create a list that has 3 tuples inside of it. Then you pass that list to dict() to convert it to a dictionary.

Accessing Dictionaries

Dictionaries claim to fame is that they are very fast. You can access any value in a dictionary via the key. If the key is not found, you will receive a KeyError.

Let’s take a look at how to use a dictionary:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict['first_name']
'James'

To get the value of first_name, you must use the following syntax: dictionary_name[key]

Now let’s try to get a key that doesn’t exist:

>>> sample_dict['address']
Traceback (most recent call last):
   Python Shell, prompt 118, line 1
builtins.KeyError: 'address'

Well that didn’t work! You asked the dictionary to give you a value that wasn’t in the dictionary!

You can use Python’s in keyword to ask if a key is in the dictionary:

>>> 'address' in sample_dict
False
>>> 'first_name' in sample_dict
True

You can also check to see if a key is not in a dictionary by using Python’s not keyword:

>>> 'first_name' not in sample_dict
False
>>> 'address' not in sample_dict
True

Another way to access keys in dictionaries is by using one of the dictionary methods. Let’s find out more about dictionary methods now!

Dictionary Methods

As with most Python data types, dictionaries have special methods you can use. Let’s check out some of the dictionary’s methods!

d.get(key[, default])

You can use the get() method to get a value. get() requires you to specify a key to look for. It optionally allows you to return a default if the key is not found. The default is None. Let’s take a look:

>>> print(sample_dict.get('address'))
None
>>> print(sample_dict.get('address', 'Not Found'))
Not Found

The first example shows you what happens when you try to get() a key that doesn’t exist without setting get‘s default. In that case, it returns None. Then the second example shows you how to set the default to the string “Not Found”.

d.clear()

The clear() method can be used to remove all the items from the dictionary.

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict
{'email': 'jdoe@gmail.com', 'first_name': 'James', 'last_name': 'Doe'}
>>> sample_dict.clear()
>>> sample_dict
{}

d.copy()

If you need to create a shallow copy of the dictionary, then the copy() method is for you:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> copied_dict = sample_dict.copy()
>>> copied_dict
{'email': 'jdoe@gmail.com', 'first_name': 'James', 'last_name': 'Doe'}

If your dictionary has objects or dictionaries inside of it, then you may end up running into logic errors due to this method as changing one dictionary can affect the copy. In those case, you should use Python’s copy module, which has a deepcopy function that will create a completely separate copy for you.

d.items()

The items() method will return a new view of the dictionary’s items:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict.items()
dict_items([('first_name', 'James'), ('last_name', 'Doe'), ('email', 'jdoe@gmail.com')])

This view object will change as the dictionary object itself changes.

d.keys()

If you need to get a view of the keys that are in a dictionary, then keys() is the method for you. As a view object, it will provide you with a dynamic view of the dictionary’s keys. You can iterate over a view and also check membership view the in keyword:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> keys = sample_dict.keys()
>>> keys
dict_keys(['first_name', 'last_name', 'email'])
>>> 'email' in keys
True
>>> len(keys)
3

d.values()

The values() method also returns a view object, but in this case it is a dynamic view of the dictionary’s values:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> values = sample_dict.values()
>>> values
dict_values(['James', 'Doe', 'jdoe@gmail.com'])
>>> 'Doe' in values
True
>>> len(values)
3

d.pop(key[, default])

Do you need to remove a key from a dictionary? Then pop() is the method for you. The pop() method takes a key and an option default string. If you don’t set the default and the key is not found, a KeyError will be raised.

Here are some examples:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict.pop('something')
Traceback (most recent call last):
   Python Shell, prompt 146, line 1
builtins.KeyError: 'something'
>>> sample_dict.pop('something', 'Not found!')
'Not found!'
>>> sample_dict.pop('first_name')
'James'
>>> sample_dict
{'email': 'jdoe@gmail.com', 'last_name': 'Doe'}

d.popitem()

The popitem() method is used to remove and return a (key, value) pair from the dictionary. The pairs are returned in last-in first-out (LIFO) order. If called on an empty dictionary, you will receive a KeyError

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict.popitem()
('email', 'jdoe@gmail.com')
>>> sample_dict
{'first_name': 'James', 'last_name': 'Doe'}

d.update([other])

Update a dictionary with the (key, value) pairs from other, overwriting existing keys. Returns None.

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict.update([('something', 'else')])
>>> sample_dict
{'email': 'jdoe@gmail.com',
'first_name': 'James',
'last_name': 'Doe',
'something': 'else'}

Modifying Your Dictionary

You will need to modify your dictionary from time to time. Let’s assume that you need to add a new key, value pair:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict['address'] = '123 Dunn St'
>>> sample_dict
{'address': '123 Dunn St',
'email': 'jdoe@gmail.com',
'first_name': 'James',
'last_name': 'Doe'}

To add a new item to a dictionary, you can use the square braces to enter a new key and set it to a value.

If you need to update a pre-existing key, you can do the following:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict['email'] = 'jame@doe.com'
>>> sample_dict
{'email': 'jame@doe.com', 'first_name': 'James', 'last_name': 'Doe'}

In this example, you set sample_dict['email'] to jame@doe.com. Whenever you set a pre-existing key to a new value, you will overwrite the previous value.

Deleting Items From Your Dictionary

Sometimes you will need to remove a key from a dictionary. You can use Python’s del keyword for that:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> del sample_dict['email']
>>> sample_dict
{'first_name': 'James', 'last_name': 'Doe'}

In this case, you tell Python to delete the key “email” from sample_dict.

The other method for removing a key is to use the dictionary’s pop() method, which was mentioned in the previous section:

>>> sample_dict = {'first_name': 'James', 'last_name': 'Doe', 'email': 'jdoe@gmail.com'}
>>> sample_dict.pop('email')
'jdoe@gmail.com'
>>> sample_dict
{'first_name': 'James', 'last_name': 'Doe'}

When you use pop(), it will return the value that is being removed.

Wrapping Up

The dictionary data type is extremely useful. You will find it handy to use for quick lookups of all kinds of data. You can set the value of the key: value pair to any object in Python. So you could store lists, tuples, or objects as values in a dictionary.

If you need a dictionary that can create a default when you go to get a key that does not exist, you should take a look at Python’s collections module. It has a defaultdict class that is made for exactly that use case.

Related Reading

• Python 101 – Learning About Tuples
• Python 101: Learning About Lists

 

The post Python 101 – Learning About Dictionaries appeared first on The Mouse Vs. The Python.

from Planet Python
via read more