Saturday, July 31, 2021

Ɓukasz Langa: Weekly Report 2021, July 26 - August 1

This week I tried to drop the open pull request count below 1,400. This was a grueling task and I barely succeeded.



from Planet Python
via read more

PyBites: Are you overwhelmed by tutorial paralysis?

Tutorial Paralysis, information overload, hoarding and never completing online courses (How large are your Udemy and Coursera libraries?)

This is the pain we’re increasingly hearing about in conversations with developers and it’s more common than you may realise.

It’s real, it’s crippling and it can make you feel like you’re going nowhere.

Even if you do manage to complete a course, is it what you needed? What next? Another course? Then what?

The reality is that nothing beats actual coding and implementationThis is where the real learning and growth happens.

It may be easier said than done, but the best time to do it is yesterday.

Tutorial Paralysis can be just as prohibiting as these signs!Photo by Sangga Rima Roman Selia on Unsplash

That’s why we recommend that anybody stuck in tutorial purgatory do the following:

1. Don’t be hard on yourself. It happens. What matters is what you do next. Also tutorials in and by themselves are not bad, they give you a good understanding of fundamental concepts, what matters is how you apply what you read.

2. Pause and reflect on what your actual goal is. Without this clear vision on what you want it’s easy to waste your time consuming content that really gets you nowhere. (Is that Blockchain tutorial really going to get you where you need to be?)

3. Make a plan on precisely what you need to do to reach that goal. Our free 6 Key Ingredients of a Successful Pythonista training (particularly day 5) can help you with this.

4. Follow the plan. Consistently. Show up every day, even when you don’t feel like it. Once you’re done, we want to see your goal and your plan. 


We’d love to hear what plan you came up with so comment below or shoot us a message. If you want help book a free assessment call using the link below.

And lastly it can be hard to go at it alone, we have a vibrant Python community a.k.a. The PyBites Slack that, at the time of this writing, has 2,700 members! You can join here and start building valuable relationships. See you there…



from Planet Python
via read more

Friday, July 30, 2021

Python Bytes: #244 vendorizing your Python podcast

<p><strong>Watch the live stream:</strong></p> <a href='https://www.youtube.com/watch?v=5Tm5tr3prPs' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by <strong>us:</strong></p> <ul> <li>Check out the <a href="https://training.talkpython.fm/courses/all"><strong>courses over at Talk Python</strong></a></li> <li>And <a href="https://pythontest.com/pytest-book/"><strong>Brian’s book too</strong></a>!</li> </ul> <p>Special guest: <strong><a href="https://twitter.com/blbraner">Brandon Braner</a></strong></p> <p><strong>Brain #1:</strong> <a href="https://pip.pypa.io/en/stable/topics/configuration/#environment-variables"><strong>pip Environmental Variables</strong></a></p> <ul> <li>The problem with snakes on a plane → no internet</li> <li>Situation: <ul> <li>want to work on some code on a plane</li> <li>project set up to use tox, which creates venvs, and pip installs dependencies <strong>from pypi.org.</strong></li> <li>but… no internet</li> </ul></li> <li>Preflight work: <ul> <li>run tox with an internet connection</li> <li>copy everything from all of the “site-packages” directories under “.tox” into a “wheels” directory or whatever you want to call it.</li> <li>set environmental variables:</li> <li>PIP_FIND_LINKS=file:/Users/okken/wheels</li> <li>PIP_NO_INDEX=1</li> <li>Try this out first BEFORE getting on the plane, to make sure you didn’t miss anything.</li> </ul></li> <li>In flight: <ul> <li>tox works fine now, using local dir as index</li> </ul></li> <li>Thanks <a href="https://twitter.com/pganssle/status/1420807532076060677?s=20">Paul Ganssle</a> for helping with this.</li> <li>All command line flags for pip are available as env variables: <ul> <li>“pip’s command line options can be set with environment variables using the format <code>PIP_[HTML_REMOVED]</code> . Dashes (<code>-</code>) have to be replaced with underscores (<code>_</code>).”</li> </ul></li> </ul> <p><strong>Michael #2: Extra, Extra, 6x Extra, hear all about it</strong></p> <ul> <li><a href="https://www.youtube.com/watch?v=lon-dEXfY2I"><strong>Michael’s Pydantic talk</strong></a></li> <li><strong><a href="https://talkpython.fm/episodes/show/327/little-automation-tools-in-python">Little Automation Tools in Python episode</a></strong></li> <li><a href="https://amzn.to/3BJ05z0"><strong>A Day in Code- Python: Learn to Code in Python through an Illustrated Story Released</strong></a> by Shari Eskenas</li> <li><a href="https://www.tabnine.com/"><strong>TabNine AI complete</strong></a></li> <li><a href="https://twitter.com/RhetTurnbull/status/1420037464366882818"><strong>macOS photos follow up</strong></a></li> <li><a href="https://github.com/mikeckennedy/jinja_partials"><strong>jinja-partials</strong></a></li> <li><a href="https://github.com/mikeckennedy/chameleon_partials"><strong>chameleon-partials</strong></a></li> <li><a href="https://github.com/mikeckennedy/fastapi-chameleon"><strong>fastapi-chameleon</strong></a></li> </ul> <p><strong>Brandon</strong> <strong>#3:</strong> <a href="https://www.kaggle.com/thirty-days-of-ml"><strong>Kaggle 30 Days of Machine Learning</strong></a> </p> <ul> <li>What is <a href="https://www.kaggle.com/">Kaggle</a> <ol> <li>Find and publish data sets</li> <li>Explore and build models in a web-based data-science environment(Jupyter Notebooks)</li> <li>Work with other data scientists and machine learning engineers</li> <li><strong>Enter competitions</strong> to solve data science challenges.</li> </ol></li> <li>Starts August 2nd, 2021</li> <li>Has an introduction to Python and covers basic and intermediate machine learning concepts</li> <li>Get some completion certificates</li> <li>Kaggle competition at the end with teams of 3</li> </ul> <p><strong>Brian #4:</strong> <a href="https://docs.github.com/en/actions/guides/building-and-testing-python"><strong>Building and testing Python with GitHub Actions</strong></a></p> <ul> <li>GitHub Docs</li> <li>This is an incredible resource</li> <li>I suggest starting with <a href="https://docs.github.com/en/actions/guides/building-and-testing-python#running-tests-with-tox"><strong>Running tests with tox</strong></a>, as it’s a super simple setup.</li> <li>And, you can test all of the tox environments locally.</li> </ul> <pre><code> name: Python package on: [push, pull_request] jobs: build: runs-on: ubuntu-latest strategy: matrix: python: [3.7, 3.8, 3.9, 3.10-dev] steps: - uses: actions/checkout@v2 - name: Setup Python uses: actions/setup-python@v2 with: python-version: $ - name: Install Tox and any other packages run: pip install tox - name: Run Tox run: tox -e py </code></pre> <p><strong>Michael #5:</strong> <a href="https://github.com/mwilliamson/python-vendorize"><strong>python-vendorize</strong></a></p> <ul> <li>via Patrick Park</li> <li>Vendoring a dependency means basically copying it into your package or your app rather than installing and importing it externally.</li> <li>Here’s a simple way to do that.</li> <li><code>python-vendorize</code> allows pure-Python dependencies to be vendorized</li> <li><code>vendorize.toml</code> might look something like:</li> </ul> <pre><code> target = "your_app/_vendor" packages = [ "six", ] </code></pre> <ul> <li>run <code>python-vendorize</code> in the same directory as <code>vendorize.toml</code></li> <li>In <code>your_app.py</code>, <code>six</code> can be imported from <code>_vendor</code>:</li> </ul> <pre><code> from ._vendor import six </code></pre> <p><strong>Brandon</strong> <strong>#6:</strong> <a href="https://supabase.io"><strong>Supabase</strong></a> <strong>and the new</strong> <a href="https://github.com/supabase/supabase-py"><strong>Python library</strong></a></p> <ul> <li>Supabase is an open source Firebase Alternative</li> <li>Postgres Database, Authentication, instant APIs, realtime subscriptions and Storage</li> <li>uses: <ul> <li><a href="https://postgrest.org/en/stable/">https://postgrest.org/en/stable/</a> for rest api access to your database</li> <li>GoTrue from Netlify for handling user authentication <a href="https://github.com/netlify/gotrue">https://github.com/netlify/gotrue</a></li> <li>Realtime client built in Elixir to listen to Postgres changes. <a href="https://github.com/supabase/realtime">https://github.com/supabase/realtime</a></li> <li>Supabase local so you don’t need internet to develop your project <a href="https://supabase.io/docs/guides/local-development">https://supabase.io/docs/guides/local-development</a></li> </ul></li> </ul> <p><strong>Extras</strong></p> <ul> <li><a href="https://simonwillison.net/2021/Jul/28/baked-data/"><strong>The Baked Data architectural pattern</strong></a> <ul> <li>Simon Willison</li> <li>“Baked Data: bundling a read-only copy of your data alongside the code for your application, as part of the same deployment”</li> </ul></li> <li><strong><a href="https://www.released.sh/">Released service - released.sh</a></strong></li> </ul> <p><strong>Joke</strong> </p> <p><a href="https://devhumor.com/media/monkeyuser-circle-of-life"><strong>The (tech) circle of life</strong></a></p>

from Planet Python
via read more

Talk Python to Me: #327: Little Automation Tools in Python

You've heard me talk to wide cast of people building amazing things with Python. Some of them are building bio-reactors to remove carbon from the air with AI and Python. Others are optimizing aerodynamics and race strategy at the highest levels of automobile racing. <br/> <br/> This episode is different. Rather than seeing how far we can push Python to the edges of technology, we are diving in to the tiny Python applications that might never be released publicly and yet can transform our day to day lives with simple automation on an individual level. <br/> <br/> We have 4 great guests with us here today: Rivers Cuomo, Jay Miller, Kim van Wyk, and Rusti Gregory. They will each share a couple of apps and the underlying packages they used to build them. I know this will be a super motivational episode for many of you. I hope that after listening, you'll transform something tedious and error-prone in your live to an instantaneous button click solution with Python.<br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Panelists</b><br/> <b>Rivers Cuomo</b>: <a href="https://twitter.com/RiversCuomo" target="_blank" rel="noopener">@RiversCuomo</a><br/> <b>Jay Miller</b>: <a href="https://twitter.com/kjaymiller" target="_blank" rel="noopener">@kjaymiller</a><br/> <b>Kim van Wyk</b>: <a href="https://twitter.com/kim_vanwyk" target="_blank" rel="noopener">@kim_vanwyk</a><br/> <b>Rusti Gregory</b>: <a href="https://ift.tt/2wwXeMF" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>DiversityOrgs.Tech</b>: <a href="https://ift.tt/3zKK5uO" target="_blank" rel="noopener">diversityorgs.tech</a><br/> <b>MP3TAG App</b>: <a href="https://ift.tt/2FFIkIa" target="_blank" rel="noopener">mp3tag.de</a><br/> <b>Screenshot from Rivers' demo clip selection app</b>: <a href="https://www.youtube.com/watch?v=DJVzQuaahSg" target="_blank" rel="noopener">Timestamp @ youtube.com</a><br/> <b>pywinauto</b>: <a href="https://ift.tt/2O1MbmK" target="_blank" rel="noopener">pywinauto.readthedocs.io</a><br/> <b>pyperclip</b>: <a href="https://ift.tt/3lb5lWw" target="_blank" rel="noopener">pypi.org</a><br/> <b>ffmpeg</b>: <a href="https://ift.tt/37cxpQY" target="_blank" rel="noopener">ffmpeg.org</a><br/> <b>selenium</b>: <a href="https://ift.tt/2bGVJ2d" target="_blank" rel="noopener">selenium-python.readthedocs.io</a><br/> <b>Github actions</b>: <a href="https://ift.tt/36rNaRz" target="_blank" rel="noopener">github.com/marketplace</a><br/> <b>RUMPS</b>: <a href="https://ift.tt/2VjQe2e" target="_blank" rel="noopener">pypi.org</a><br/> <b>py2app</b>: <a href="https://ift.tt/2ZgBOA8" target="_blank" rel="noopener">pypi.org</a><br/> <b>PyMuPDF</b>: <a href="https://ift.tt/3fS6RHd" target="_blank" rel="noopener">pypi.org</a><br/> <b>Gooey</b>: <a href="https://ift.tt/2VnwKd2" target="_blank" rel="noopener">pypi.org</a><br/> <b>conduit podcast</b>: <a href="https://ift.tt/3i8MyrC" target="_blank" rel="noopener">relay.fm</a><br/> <b>feedparser</b>: <a href="https://ift.tt/3ld2f4a" target="_blank" rel="noopener">feedparser.readthedocs.io</a><br/> <b>awesome-python audio recommendations</b>: <a href="https://ift.tt/3zRTH6X" target="_blank" rel="noopener">awesome-python.com</a><br/> <b>foxdot live coding</b>: <a href="https://ift.tt/3C20s7Y" target="_blank" rel="noopener">toplap.org</a><br/> <b>spotipy</b>: <a href="https://ift.tt/37a4X2d" target="_blank" rel="noopener">pypi.org</a><br/> <b>pipx</b>: <a href="https://ift.tt/34n1zjj" target="_blank" rel="noopener">github.com</a><br/> <b>eyed3</b>: <a href="https://ift.tt/3xbTObL" target="_blank" rel="noopener">eyed3.readthedocs.io</a><br/> <b>Youtube Live Stream</b>: <a href="https://www.youtube.com/watch?v=DJVzQuaahSg" target="_blank" rel="noopener">youtube.com</a><br/> <br/> <b>Episodes referenced at the beginning:</b><br/> <b>Bio-reactor</b>: <a href="https://ift.tt/2OAYNiY" target="_blank" rel="noopener">talkpython.fm</a><br/> <b>Nascar</b>: <a href="https://ift.tt/33d5NsH" target="_blank" rel="noopener">talkpython.fm</a><br/> <b>F1</b>: <a href="https://ift.tt/2KPLw79" target="_blank" rel="noopener">talkpython.fm</a><br/> <b>Episode transcripts</b>: <a href="/episodes/transcript/327/little-automation-tools-in-python" target="_blank" rel="noopener">talkpython.fm</a><br/></div><br/> <strong>Sponsors</strong><br/> <br/> <a href='https://ift.tt/3aBjB2k> <a href='https://ift.tt/2PVc9qH Python Training</a><br> <a href='https://ift.tt/3x0z72P>

from Planet Python
via read more

Mike Driscoll: Pre-Order Automating Excel with Python

My 10th Python book is called Automating Excel with Python: Processing Spreadsheets with OpenPyXL. It only has 11 more days to go on Kickstarter where you can get an exclusive t-shirt!

I also have the eBook available for pre-order on Gumroad. No matter whether you purchase a copy on Kickstarter or Gumroad, you will be getting an early version of the book plus all the updates that I ever make to the book.

You can get some sample chapters here so you can try out the book before you commit to purchasing a copy!

Automating Excel with Python

In this book, you will learn how to use Python to do the following:

  • Create Excel spreadsheets
  • Read Excel spreadsheets
  • Create different cell types
  • Add and remove sheets
  • Convert Excel spreadsheets into other file types
  • Cell styling (changing fonts, inserting images, background colors, etc)
  • Conditional formattings
  • Adding charts
  • and much more!

The post Pre-Order Automating Excel with Python appeared first on Mouse Vs Python.



from Planet Python
via read more

How to Keep Track of TensorFlow/Keras Model Development with Neptune

The model development lifecycle starts with data exploration, then we choose features for our model, choose a baseline algorithm, and next, we try to improve baseline performance with different algorithms and parameter tuning. Sounds simple enough. But, during all of this, you’ll probably create multiple notebooks, or modify one notebook over and over again. This […]

The post How to Keep Track of TensorFlow/Keras Model Development with Neptune appeared first on neptune.ai.



from Planet SciPy
read more

Stack Abuse: How to Join/Concatenate/Append Strings in Python

Introduction

In this short tutorial, we'll take a look at how to concatenate strings in Python, through a few different approaches.

It's worth noting that strings in Python are immutable - a string object in memory cannot be changed once created:

newString = "Some new string"

If you'd like to change this string in any way - under the hood, a new string with those changes is created. The same applies to concatenating strings - a new object must be created in memory.

String Concatenation and String Appending

Concatenation of strings refers to joining two or more strings together, as if links in a chain. You can concatenate in any order, such as concatenating str1 between str2 and str3.

Appending strings refers to appending one or more strings to the end of another string.

In some cases, these terms are absolutely interchangeable. Appending strings is the same operation as concatenating strings at the end of another. Let's begin with the simplest way of concatenating/appending two (or more) strings.

Concatenate or Append Strings with The + Operator

In Python, a string is a list of characters, which means that the + operator can be used to add their constituent elements together in a new list:

title = "Prof. "
name = "Richard Feynman"

result = title + name
print(result)

This results in:

Prof. Richard Feynman

This operator doesn't limit the amount of strings which can be added together, so you can easily join a large number of strings:

string1 = "Concatenating"
string2 = "strings"
string3 = "in Python"
string4 = "is easy!"

print(string1 + string2 + string3 + string4)

Though, if your aim is to construct a sentence from a list of strings such as this, concatenating them manually, and without special characters is both inefficient and produces an unintelligible output:

Concatenatingstringsin Pythonis easy!

It'd be much more sensible to iterate through a list of strings, and add them together with a whitespace between each concatenated string:

strings = ["Concatenating", "strings", "in Python", "is easy!"]

result = ""
for string in strings:
    result += string + " "

print(result)

This results in:

Concatenating strings in Python is easy! 

A shorthand operator you can use to concatenate two strings is +=, just like in the previous example. This saves you from the trouble of having to create a new variable to store the results, as you can reuse one of the existing reference variables to assign to the new object in memory:

string1 = "one"
string2 = "two"

string1 += string2
print(string1) # Output: onetwo

Limitation of the + Operator

The main limitation of the + operator is the fact that you can't mix-and-match types. You can't add an integer to a list of characters, for instance:

print("some string" + 2)

Many languages, such as JavaScript and Java pick up on this, and would automatically convert the integer into a string (match the types) and perform the concatenation, though, Python would throw a TypeError:

TypeError: can only concatenate str (not "int") to str

However, there is an easy workaround - you can use Python's built-in str() function which converts compatibile data types into a string. Let's add some integers to our strings list and box all of the items with a str() in case there are non-string elements:

strings = ["Concatenating", "strings", "in Python", "is easy!", 5, 2]

result = ""
for string in strings:
    result += str(string) + " "

print(result)

This results in:

Concatenating strings in Python is easy! 5 2 

Concatenate or Append Strings with The * Operator

If you want to create a new string by replicating a string n amount of times and appending it, you can achieve this with the * operator:

string = "w"

print(string * 3) # Output: www

This can be even more useful when combined with the + operator:

print("w"*3 + "." + "stackabuse.com")

Which results in:

www.stackabuse.com

Concatenate or Append Strings with The % Operator

Again, concatenation doesn't necessarily mean we're adding a string to the end. Using the % operator, we can perform string interpolation. By adding % in a string as a marker, we can replace the markers with concrete strings later on:

string = "This %s is a %s string" % ("here", "random")

print(string)

This should output:

This here is a random string

Similarly enough, you can use other markers for other data types:

  • %d - for integer representation:
string = "This is a string%d" % (1)
print(string) # Output: This is a string
  • %c - for character representation:
string = "This string starts with a %c" % ('T')
print(string) # Output: This string starts with a T
  • %f - for floating number:
string = "There is a %f percent chance that you'll learn string concatenation in Python after reading this article" % (99.99)
print(string) # Output filled 99,990000 in %f place

Note: If you wish to explicitly mark how many digits should the number be rounded to (say 2), you can achieve it with: %.2f.

If you'd like to read more about formatting strings in Python and the different ways to do it, read our Guide to Formatting Strings in Python 3's f-Strings.

Concatenating Strings With the join() Method

The join() method takes an iterable as an argument and returns a string created by joining the elements of that iterable. It's worth noting that these have to be strings - each element is not inherently converted using str(), unlike our own method from before.

Additionally, a separator is used to define the separator between the joined strings, and it's the base string we call join() on:

my_list = ["1", "2", "3", "4"] # List - iterable
string_from_list = "-".join(my_list) # The separator is "-"

print(string_from_list)

This should output:

1-2-3-4

In a lot of cases, the separator is just a whitespace, so you'll commonly be seeing:

" ".join(iterable)

Implementing a Custom join() Method

Since the built-in join() method might behave a bit differently than you might've expected, let's implement our own join() method with an adjustable separator.

We'll want it to be able to handle 2D lists as well, so if a list contains another list within itself - it's flattened to a 1-dimensional list, before joined:

import itertools

def join(iterable, separator):
    # Empty string to hold result
    result = ""
    # Flatten 2D potentially 2D list into 1D
    iterable = list(itertools.chain(*iterable))
    # Concatenate strings with separator
    for string in iterable:
        result += str(string) + separator
    return result
    

string = join(['a', 'b', 'c'], ' ')
string2 = join(['a', ['b', 'c'], 'd'], ' ')

print(string)
print(string2)

This results in:

a b c 
a b c d 

Concatenating Strings Using Space

A simple way to concatenate strings, usually only when you're printing them is to leverage the space bar. This approach is commonly used only for printing, since assigning it to an object in memory is easy, but awkward:

print("Concat" " strings" " using Space")

If you'd like to avoid using whitespaces in the strings, you can add commas (,) between each element:

print("Concat", "strings", "using Space")

Both of these result in:

Concat strings using Space

If you'd like to assign them to a variable, you're free to do so, and they'll automatically be concatenated into a single string:

string = "this " "is " "a " "big " "string"
print(type(string))
print(string)

This results in:

<class 'str'>
this is a big string

You can even do multi-line strings. To achieve this, we add a \ to the end of each line to let Python know there's more than one line:

multi_line_string = "this " "string " \
                                  "is coded in " "three lines " \
                          "but printed in one"
                                  
print(multi_line_string)

This results in:

this string is coded in three lines but printed in one

Though, this approach is fiddly and awkward, and others are preferred.

Note: The same effect cannot be achieved with variables, only string literals:

string1 = "one string"
string2 = "two string"

final_string = string1 string2

This results in:

File "<string>", line 4
    final_string = string1 string2
                           ^
SyntaxError: invalid syntax

Conclusion

Joining/Appending/Concatenating Strings in Python is fairly simple and like everything related to Python, there are many ways to do it. It just comes down to your needs and preferences.

In this short tutorial, we've taken a look at some of the ways to concatenate strings



from Planet Python
via read more

Real Python: The Real Python Podcast – Episode #71: Start Using a Debugger With Your Python Code

Are you still sprinkling print statements throughout your code while writing it? Print statements are often clunky and offer only a limited view of the state of your code. Have you thought there must be a better way? This week on the show, we have Nina Zakharenko to discuss her conference talk titled "Goodbye Print, Hello Debugger."


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]



from Planet Python
via read more

The Real Python Podcast – Episode #71: Start Using a Debugger With Your Python Code

Are you still sprinkling print statements throughout your code while writing it? Print statements are often clunky and offer only a limited view of the state of your code. Have you thought there must be a better way? This week on the show, we have Nina Zakharenko to discuss her conference talk titled "Goodbye Print, Hello Debugger."


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]



from Real Python
read more

Thursday, July 29, 2021

State of Data Science 2021: Becoming “Essential,” Though Untapped Potential Remains

Among the many new concepts introduced to us throughout 2020 was the idea of the “essential worker.” What jobs or roles are so integral to the functioning of a business that the company wouldn’t be able to operate without them? The idea of “essentialness” provides a helpful framework to consider how effectively data science has integrated with an organization’s most important operations.

from Planet SciPy
read more

Predicting Stock Prices Using Machine Learning

The stock market is known for being volatile, dynamic, and nonlinear. Accurate stock price prediction is extremely challenging because of multiple (macro and micro) factors, such as politics, global economic conditions, unexpected events, a company’s financial performance, and so on.  But, all of this also means that there’s a lot of data to find patterns […]

The post Predicting Stock Prices Using Machine Learning appeared first on neptune.ai.



from Planet SciPy
read more

Python for Beginners: Convert a List to String in Python

Python strings are one of the most commonly used data types. Whereas, Python lists are the most commonly used data structures. In this article, we will try to convert a list to a string using different functions in python. We will use string methods like join() and functions like map() and str() to convert a list into string.

List to string using String Concatenation

The easiest way to convert a list to a string in python is by using for loop and string concatenation. To convert a list to a string, we can simply convert each element of the list to a string. Then we can concatenate them to form a string.

To perform the conversion, first we will take an empty string. After that, we will start adding each element of the list to the empty string after converting them to string. In this process, we will also add spaces between each element. This can be done as follows.

myList = ["PFB", 11.2, 11, "Python"]
print("The List is:",myList)
myString = ""
for elem in myList:
    myString = myString + str(elem) + " "

print("Output String is:")
print(myString.rstrip())

Output:


The List is: ['PFB', 11.2, 11, 'Python']
Output String is:
PFB 11.2 11 Python

In the above example, we can see that an extra space has been added at the end of the string. We can use the rstrip() method to remove the extra space from the string. 

An alternative way to convert a list to string in python is to use the join() method. The join() method is used to create a string from a list of strings.

The join() method is invoked on a separator which is used to separate the elements of the list in the string. The list of strings is given as input to the join() method and It returns a string created by the elements of the list.

We will create an empty string to which all the elements of the list will be concatenated. To create the string, we will take each element of the list one by one and will convert it into string. Then, we will create a list of  strings using the string made by previous elements of the list and the current element. We will create the new string from the current string and the previously made string using the join() method by taking a space character as a separator and invoking the join() method on it. We will do this process on all the elements of the list using a for loop till the complete string is formed. This can be seen in the following example.

myList = ["PFB", 11.2, 11, "Python"]
print("The List is:", myList)
myString = ""
for elem in myList:
    myString = " ".join([myString, str(elem)])

print("Output String is:")
print(myString.lstrip())

Output:

The List is: ['PFB', 11.2, 11, 'Python']
Output String is:
PFB 11.2 11 Python

In the above method, an extra space is added at the left of the output string which has to be removed using the lstrip() method. To avoid this,Instead of applying str() function on every element of the list, we can use map() function  to convert each element of the list into a string so that we can perform the string concatenation using join() method to get the output string using a single statement.

The map() function takes as input a function and an iterable as argument and executes the function on each element of the iterable object and returns  the output map object which can be converted into a list.

To convert the elements of the list to strings, we will pass the str() function and the input list to the map() method. After that, we can create a string from a list of strings using the join() method as follows.

myList = ["PFB", 11.2, 11, "Python"]
print("The List is:", myList)
strList = list(map(str, myList))
myString = " ".join(strList)

print("Output String is:")
print(myString.lstrip())

Output:

The List is: ['PFB', 11.2, 11, 'Python']
Output String is:
PFB 11.2 11 Python

Using List Comprehension to convert a list to a string

We can use list comprehension to convert a list into a string. For this, We will convert each element of the list to string using list comprehension and str() function and then we can concatenate them using the join() method as follows.

myList = ["PFB", 11.2, 11, "Python"]
print("The List is:", myList)
strList = [str(i) for i in myList]
myString = " ".join(strList)

print("Output String is:", myString)

Output:

The List is: ['PFB', 11.2, 11, 'Python']
Output String is: PFB 11.2 11 Python

Alternatively, we can use the map() function to convert the elements of the list to string. Then we will create a new list from the map object created from the map() function using list comprehension and will give it as an input to the join() method to create a string from the list as follows.

myList = ["PFB", 11.2, 11, "Python"]
print("The List is:", myList)
myString = " ".join([i for i in map(str, myList)])
print("Output String is:", myString)

Output:

The List is: ['PFB', 11.2, 11, 'Python']
Output String is: PFB 11.2 11 Python

Conclusion

In this article, we have used the str(), map() and join() functions to convert a list to string. We have also seen how to use list comprehension for this task. To know how to convert a string into a list, read this article on string split operation in python.

The post Convert a List to String in Python appeared first on PythonForBeginners.com.



from Planet Python
via read more

Twisted Matrix Labs: Twisted 21.7.0 Released

 On behalf of the Twisted contributors I announce the final release of Twisted 21.7.0

This is mostly a bugfix release.

Python 3.5 is no longer a supported platform.
The minimum supported platform is Python 3.6.7.

The notable features are:
  • Python 3.10 beta is now a supported platform and should be ready for the final 3.10 release.
  • twisted.web.template.renderElement() now accepts any IRequest implementer instead of only twisted.web.server.Request. Add type hints to twisted.web.template. (#10184)
  • Type hinting was added to twisted.internet.defer, making this the first release  of Twisted where you might reasonably be able to use mypy without your own custom stub files. (#10017)
The full release notes are available at


Documentation is available at


Wheels for the release candidate are available on PyPI


    python -m pip install Twisted==21.7.0

Many thanks to everyone who had a part in Twisted - the supporters of the Twisted Software Foundation, the developers, and all the people testing and building great things with Twisted!

Enjoy the release

-Adi Roiban


from Planet Python
via read more

Wednesday, July 28, 2021

Lucas Cimon: Bonnes pratiques Gitlab CI

Logo Gitlab

À oui.sncf, je travaille au sein d'une Ă©quipe en charge de l'usine logicielle, qui administre depuis des annĂ©es une instance Gitlab self-hosted.

Cet article contient quelques-unes de nos recommandations Ă  l'intention des utilisateurs de notre Gitlab, ayant pour but Ă  la fois amĂ©liorer les performances de leurs pipelines …

Permalink



from Planet Python
via read more

PyCharm: Introducing PyCharm 2021.2!

This release comes with Python 3.10 support, auto-reload for browser HTML preview, and more!

Before we start the overview of the major PyCharm 2021.2 features, we have some important information to make you aware of.

Complete_current_statement

First of all, we have good news for our Asian users. Starting with this version, you can enjoy a fully localized UI in Chinese, Korean, or Japanese. Localization is available as a non-bundled language pack plugin, which can be easily installed in your IDE.

Secondly, please note that we are planning to end support for several packages. Namely for mako, buildout, web2py. This change will be introduced later this year, starting from PyCharm 2021.3.

DOWNLOAD PYCHARM 2021.2

Python 3.10

Structural Pattern Matching is coming in Python 3.10. PyCharm provides a number of key features to help you adjust to its arrival, like the Unused local symbols and Unreachable code inspections, smart code completion and syntax highlighting for the match and case keywords, and the Complete Current Statement action.

Complete_current_statement

In Python 3.10 you will be able to use int | str for union types instead of Union[int, str]. This functionality is already available in earlier versions of Python through the use of from __future__ import annotations.

Union_types

PyCharm provides overall code insight for the new syntax, including intention actions and information in the Quick Documentation. PyCharm now supports type inference for isinstance and issubclass arguments with the new syntax type.

Python console support in collaborative mode

Working in the Python console is now supported on both the client machine and the host side. During a collaborative coding session, guests can review the code while the host runs it in the interactive console. Support for the Python console in Code With Me comes complete with all the usual PyCharm features, such as on-the-fly syntax highlighting with inspections and code completion.

Python_console

DOWNLOAD PYCHARM 2021.2

Databases [Pro only]

Code completion for fields and operators in the MongoDB console

Now PyCharm can complete fields, nested fields, and fields inside aggregation expressions, as well as query operators, projection operators, update operators, and aggregation stages in the MongoDB console.
mongodb

Context live templates from the data editor with SQL scripts action

Context live templates now work directly from the data editor. If you’re working with a table and you wish to query it, you can easily do so with the help of the SQL scripts action!

live_templates_sql

DOWNLOAD PYCHARM 2021.2

Frontend development [Pro only]

Reload pages in browser on save

PyCharm lets you preview HTML files in a browser using the built-in web server. Now it is able to automatically update the pages in a browser as you edit and save your HTML, CSS, and JavaScript files. To get started, open an HTML file in the editor, hover over the corner of the editor, and click on the icon for the browser you want to use – all browsers are supported.

html_preview

Onboarding Tour in the Features Trainer plugin

You can now take the new Onboarding Tour in the IDE Features Trainer plugin.
With this 7-minute tour, you can get familiar with the PyCharm user interface and learn how to code in Python with smart assistance! You can start the tour from the Welcome screen or from the PyCharm main menu: Help | Learn IDE Features.

onboarding_tour

DOWNLOAD PYCHARM 2021.2

Testing

Test runner auto-detection

PyCharm auto-detects when there is a test runner installed on your Python interpreter and uses it to run tests. If no specific test runner is installed, PyCharm uses unittest. Still, you always have an option to explicitly specify the required test runner in the project settings. To explicitly set the required test runner in the project settings, press ⌘, to open the IDE settings and select Tools | Python Integrated Tools, and then select the target test runner from the Default test runner list.

Integrate with TestRail

The Test Management plugin now includes support for Python and is available for PyCharm Community. The Test Management plugin helps keep your unit tests in sync with test cases in TMS. You can load test cases from TestRail and link them with existing and new unit tests.
When you copy and paste a test case to a Python file a copy of the unit test will be pasted. You can customize the unit test template and share it with the team. With the plugin you can find test cases that are not yet automated (not linked with unit tests) and obsolete tests.

test-runner

Generate Test Data

Being able to insert a randomly generated email, name, or phone number is important when developing unit tests. As a part of our Test Automation Kit the new Test Data plugin plugin brings a lot of useful actions that help to generate random data. Use the Generate menu (Cmd+N) to see all available options. If you need a specific format, feel free to create your own custom data format based on regular expression or Velocity templates. All custom data types are available in bulk mode and can be shared with your team.

test-runner

DOWNLOAD PYCHARM 2021.2

In this blog post we covered only a few major features of PyCharm 2021.2. To learn about all the other new features in PyCharm 2021.2 visit our What’s New page.



from Planet Python
via read more

Python⇒Speed: Saving memory with Pandas 1.3's new string dtype

When you’re loading many strings into Pandas, you’re going to use a lot of memory. If you have only a limited number of strings, you can save memory with categoricals, but that’s only helpful in a limited number of situations.

With Pandas 1.3, there’s a new option that can save memory on large number of strings as well, simply by changing to a new column type. Let’s see how.

Read more...

from Planet Python
via read more

10 NLP Projects to Boost Your Resume

Natural Language Processing (NLP) is a very exciting field. Already, NLP projects and applications are visible all around us in our daily life. From conversational agents (Amazon Alexa) to sentiment analysis (Hubspot’s customer feedback analysis feature), language recognition and translation (Google Translate), spelling correction (Grammarly), and much more.  Whether you’re a developer or data scientist […]

The post 10 NLP Projects to Boost Your Resume appeared first on neptune.ai.



from Planet SciPy
read more

Python for Beginners: Convert Integer to String in Python

Python strings are one of the most used data types while doing data analysis for operations like pattern matching. In this article, we will use different ways to convert an integer to string in python.

Convert integer to string using str() function

The easiest way to convert an integer to string is to use the str() function. The str() function takes the integer as input and returns its string representation as follows.

myInt = 1117
myStr = str(myInt)
print("The integer myInt is:", myInt)
print("The string myStr is:", myStr)

Output:

The integer myInt is: 1117
The string myStr is: 1117

We can check the type of input variable and output variable to confirm if the integer has been  converted to a string or not. To do this, we will use the type() function. The type function takes a python object as input and returns the data type of the input object as follows.

myInt = 1117
myStr = str(myInt)
print("The data type of myInt is:", type(myInt))
print("The data type of myStr is:", type(myStr))

Output:

The data type of myInt is: <class 'int'>
The data type of myStr is: <class 'str'>

Convert integer to string using string formatting

String formatting is a method to insert a variable or another string to a predefined string. We can also use string formatting to convert an integer to string. We will use the “%s” operator as well as the format() method to convert an integer to string in this article.

The “%s” operator is used to format a value inside a string. It is generally used to avoid string concatenation. But, we can use this operator to convert an integer to a string. For this, first we will create an empty string and put a %s placeholder in the empty string. After that, we can specify the integer which has to be converted into string. During execution of the program, the python interpreter will convert the integer into string as seen in the following example.

myInt = 1117
myStr = "%s" % myInt
print("myInt is:",myInt)
print("The data type of myInt is:", type(myInt))
print("myStr is:",myStr)
print("The data type of myStr is:", type(myStr))

Output:

myInt is: 1117
The data type of myInt is: <class 'int'>
myStr is: 1117
The data type of myStr is: <class 'str'>

In place of the “%s” operator, we can also use format() method to perform the conversion. For this we can put a {} placeholder in an empty string. After that, we can invoke the format method on the empty string with the integer given as the input to the format() method. This will convert the integer into string as follows.

myInt = 1117
myStr = "{}".format(myInt)
print("myInt is:",myInt)
print("The data type of myInt is:", type(myInt))
print("myStr is:",myStr)
print("The data type of myStr is:", type(myStr))

Output:

myInt is: 1117
The data type of myInt is: <class 'int'>
myStr is: 1117
The data type of myStr is: <class 'str'>

Conversion using F-strings

F strings are used to embed a value or an expression into a string. We can also use f strings to convert an integer into a string. 

The syntax for using f strings is similar to that of format() method. The only difference is that we can put the variables directly into the placeholders. This makes the code more readable. To put an integer variable n into a string, we simply put n into the {} placeholder as follows.

f"This is a string containing {n}"

To convert an integer to string using f strings, we will declare an empty string with only a single placeholder for the integer. In this way, at runtime, the integer will be converted to string. This can be seen in the following example.

myInt = 1117
myStr = f"{myInt}"
print("myInt is:",myInt)
print("The data type of myInt is:", type(myInt))
print("myStr is:",myStr)
print("The data type of myStr is:", type(myStr))

Output:

myInt is: 1117
The data type of myInt is: <class 'int'>
myStr is: 1117
The data type of myStr is: <class 'str'>

Conclusion

In this article, we have seen different ways to convert an integer into a string in python. We have used in-built str() method, string formatting as well as f-strings. To read more about strings, you can read this article on python string split operation.We can also write the programs used in this article with exception handling using python try except to make the programs more robust and handle errors in a systematic way.  

The post Convert Integer to String in Python appeared first on PythonForBeginners.com.



from Planet Python
via read more

Real Python: Python and REST APIs: Interacting With Web Services

There’s an amazing amount of data available on the Web. Many web services, like YouTube and GitHub, make their data accessible to third-party applications through an application programming interface (API). One of the most popular ways to build APIs is the REST architecture style. Python provides some great tools not only to get data from REST APIs but also to build your own Python REST APIs.

In this tutorial, you’ll learn:

  • What REST architecture is
  • How REST APIs provide access to web data
  • How to consume data from REST APIs using the requests library
  • What steps to take to build a REST API
  • What some popular Python tools are for building REST APIs

By using Python and REST APIs, you can retrieve, parse, update, and manipulate the data provided by any web service you’re interested in.

Free Bonus: Click here to download a copy of the "REST API Examples" Guide and get a hands-on introduction to Python + REST API principles with actionable examples.

REST Architecture

REST stands for representational state transfer and is a software architecture style that defines a pattern for client and server communications over a network. REST provides a set of constraints for software architecture to promote performance, scalability, simplicity, and reliability in the system.

REST defines the following architectural constraints:

  • Stateless: The server won’t maintain any state between requests from the client.
  • Client-server: The client and server must be decoupled from each other, allowing each to develop independently.
  • Cacheable: The data retrieved from the server should be cacheable either by the client or by the server.
  • Uniform interface: The server will provide a uniform interface for accessing resources without defining their representation.
  • Layered system: The client may access the resources on the server indirectly through other layers such as a proxy or load balancer.
  • Code on demand (optional): The server may transfer code to the client that it can run, such as JavaScript for a single-page application.

Note, REST is not a specification but a set of guidelines on how to architect a network-connected software system.

REST APIs and Web Services

A REST web service is any web service that adheres to REST architecture constraints. These web services expose their data to the outside world through an API. REST APIs provide access to web service data through public web URLs.

For example, here’s one of the URLs for GitHub’s REST API:

https://api.github.com/users/<username>

This URL allows you to access information about a specific GitHub user. You access data from a REST API by sending an HTTP request to a specific URL and processing the response.

HTTP Methods

REST APIs listen for HTTP methods like GET, POST, and DELETE to know which operations to perform on the web service’s resources. A resource is any data available in the web service that can be accessed and manipulated with HTTP requests to the REST API. The HTTP method tells the API which action to perform on the resource.

While there are many HTTP methods, the five methods listed below are the most commonly used with REST APIs:

HTTP method Description
GET Retrieve an existing resource.
POST Create a new resource.
PUT Update an existing resource.
PATCH Partially update an existing resource.
DELETE Delete a resource.

A REST API client application can use these five HTTP methods to manage the state of resources in the web service.

Status Codes

Once a REST API receives and processes an HTTP request, it will return an HTTP response. Included in this response is an HTTP status code. This code provides information about the results of the request. An application sending requests to the API can check the status code and perform actions based on the result. These actions could include handling errors or displaying a success message to a user.

Below is a list of the most common status codes returned by REST APIs:

Code Meaning Description
200 OK The requested action was successful.
201 Created A new resource was created.
202 Accepted The request was received, but no modification has been made yet.
204 No Content The request was successful, but the response has no content.
400 Bad Request The request was malformed.
401 Unauthorized The client is not authorized to perform the requested action.
404 Not Found The requested resource was not found.
415 Unsupported Media Type The request data format is not supported by the server.
422 Unprocessable Entity The request data was properly formatted but contained invalid or missing data.
500 Internal Server Error The server threw an error when processing the request.

These ten status codes represent only a small subset of the available HTTP status codes. Status codes are numbered based on the category of the result:

Read the full article at https://realpython.com/api-integration-in-python/ »


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]



from Planet Python
via read more

Python and REST APIs: Interacting With Web Services

There’s an amazing amount of data available on the Web. Many web services, like YouTube and GitHub, make their data accessible to third-party applications through an application programming interface (API). One of the most popular ways to build APIs is the REST architecture style. Python provides some great tools not only to get data from REST APIs but also to build your own Python REST APIs.

In this tutorial, you’ll learn:

  • What REST architecture is
  • How REST APIs provide access to web data
  • How to consume data from REST APIs using the requests library
  • What steps to take to build a REST API
  • What some popular Python tools are for building REST APIs

By using Python and REST APIs, you can retrieve, parse, update, and manipulate the data provided by any web service you’re interested in.

REST Architecture

REST stands for representational state transfer and is a software architecture style that defines a pattern for client and server communications over a network. REST provides a set of constraints for software architecture to promote performance, scalability, simplicity, and reliability in the system.

REST defines the following architectural constraints:

  • Stateless: The server won’t maintain any state between requests from the client.
  • Client-server: The client and server must be decoupled from each other, allowing each to develop independently.
  • Cacheable: The data retrieved from the server should be cacheable either by the client or by the server.
  • Uniform interface: The server will provide a uniform interface for accessing resources without defining their representation.
  • Layered system: The client may access the resources on the server indirectly through other layers such as a proxy or load balancer.
  • Code on demand (optional): The server may transfer code to the client that it can run, such as JavaScript for a single-page application.

Note, REST is not a specification but a set of guidelines on how to architect a network-connected software system.

REST APIs and Web Services

A REST web service is any web service that adheres to REST architecture constraints. These web services expose their data to the outside world through an API. REST APIs provide access to web service data through public web URLs.

For example, here’s one of the URLs for GitHub’s REST API:

https://api.github.com/users/<username>

This URL allows you to access information about a specific GitHub user. You access data from a REST API by sending an HTTP request to a specific URL and processing the response.

HTTP Methods

REST APIs listen for HTTP methods like GET, POST, and DELETE to know which operations to perform on the web service’s resources. A resource is any data available in the web service that can be accessed and manipulated with HTTP requests to the REST API. The HTTP method tells the API which action to perform on the resource.

While there are many HTTP methods, the five methods listed below are the most commonly used with REST APIs:

HTTP method Description
GET Retrieve an existing resource.
POST Create a new resource.
PUT Update an existing resource.
PATCH Partially update an existing resource.
DELETE Delete a resource.

A REST API client application can use these five HTTP methods to manage the state of resources in the web service.

Status Codes

Once a REST API receives and processes an HTTP request, it will return an HTTP response. Included in this response is an HTTP status code. This code provides information about the results of the request. An application sending requests to the API can check the status code and perform actions based on the result. These actions could include handling errors or displaying a success message to a user.

Below is a list of the most common status codes returned by REST APIs:

Code Meaning Description
200 OK The requested action was successful.
201 Created A new resource was created.
202 Accepted The request was received, but no modification has been made yet.
204 No Content The request was successful, but the response has no content.
400 Bad Request The request was malformed.
401 Unauthorized The client is not authorized to perform the requested action.
404 Not Found The requested resource was not found.
415 Unsupported Media Type The request data format is not supported by the server.
422 Unprocessable Entity The request data was properly formatted but contained invalid or missing data.
500 Internal Server Error The server threw an error when processing the request.

These ten status codes represent only a small subset of the available HTTP status codes. Status codes are numbered based on the category of the result:

Read the full article at https://realpython.com/api-integration-in-python/ »


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]



from Real Python
read more

Stack Abuse: Observer Design Pattern in Python

Introduction

Software Design Patterns help accelerate the development process by providing a reusable blueprint for your code to solve a particular problem. We follow Design Patterns to write generalized, reusable, and readable code that could be easily understood by others familiar with the patterns we've applied.

They encapsulate cumulative experience of software engineers solving the same problems, and represent solutions to common design-related issuees.

There are different classifications of design patterns depending on which class of problems they solve - among which the Observer Design Pattern belongs to the Behavioral Pattern class.

This class of patterns determines how objects communicate with each other. In this guide, you will learn everything you need to know about the Observer Design Pattern and understand how we can use it to solve certain problems efficiently.

Observer Design Pattern

The Observer Design Pattern deals with One-to-Many relationships and utilizes events to let subscribed entities know about changes in an observable.

The source of these events is called the subject or observable which sends events as streams. The observers or sinks can subscribe to the observable to obtain the events. The observable keeps track of the list of observers and notifies them of the changes when the state of the observable changes.

Architecture Diagram of Observer Design Pattern - shows a news feed as an observable and multiple readers as observers subscribed to the news feed

This functionality has many implications and implementations, and similar functionality is all around you. It's an extremely simple, yet very effective and wide-spread pattern.

A similar implementation of this design pattern is seen in generating feeds on your social platforms - the Pub/Sub (Publisher/Subscriber) Model/Pattern. When a content publisher publishes their posts, the subscribers get notified of the content. A similar analogy may be people looking out for a flare signal or a firework for a certain event, and reacting (or not) depending on their specific roles.

Does that mean that the Observer Design Pattern and Publish/Subscribe Pattern are the same?

Previously, both patterns were synonymous. Nowadays, each pattern has distinct traits that make them two separate patterns.

The following are the major differences between the Observer Pattern and the Pub/Sub Pattern:

  • Observers and Subjects are tightly coupled. The subjects must keep track of their observers. Whereas in the Pub/Sub pattern, they are loosely coupled with a message queue in between observers and subjects.
  • The events are passed in a synchronous manner from the Subjects to the Observers. But in Pub/Sub patterns, the events are passed asynchronously.
  • In the Observer pattern, both the Subjects and Observers reside on the same application locality whereas they can reside on different localities in the Pub/Sub pattern.

One of the best ways to get a feel for this pattern is to implement it, let's implement it in Python!

Implementation

A basic implementation requires two classes - an Observable and an Observer. The Observer class is initialized with an object as an argument. The object is none other than an Observable to keep track of, to which it is subscribed upon creation.

The class also has a notify() function which triggers a reaction and acknowledges the receival of a notification/event from the observable:

class Observer:

    def __init__(self, observable):
        observable.subscribe(self)

    def notify(
        self,
        observable,
        *args,
        **kwargs
        ):
        print ('Got', args, kwargs, 'From', observable)

The Observable class is initialized with an empty list to hold the Observer instances. It also has functions such as subscribe() to add an observer, notify_observers() to call the notify() function on each observer, and unsubscribe() to remove the observer from the list:

class Observable:

    def __init__(self):
        self._observers = []

    def subscribe(self, observer):
        self._observers.append(observer)

    def notify_observers(self, *args, **kwargs):
        for obs in self._observers:
            obs.notify(self, *args, **kwargs)

    def unsubscribe(self, observer):
        self._observers.remove(observer)

Plugging in all of the above-mentioned components, let's write some code that sets up an observer and observable and sends messages, which triggers a reaction:

# observer_pattern.py

"""
Demonstrating the Observer pattern implementation
"""
# Initializing the subject
subject = Observable()

# Initializing twp observers with the subject object
observer1 = Observer(subject)
observer2 = Observer(subject)

# The following message will be notified to 2 observers
subject.notify_observers('This is the 1st broadcast',
                         kw='From the Observer')
subject.unsubscribe(observer2)

# The following message will be notified to just 1 observer since
# the observer has been unsubscribed
subject.notify_observers('This is the 2nd broadcast',
                         kw='From the Observer')

Notice that we also unsubscribe an observer before publishing the second message. This will lead to the message being printed only once instead of twice on the second attempt, as it is received by only one subscriber.

Running this code will result in:

$ python observer_pattern.py
Got ('This is the 1st broadcast',) {'kw': 'From the Observer'} From <__main__.Observable object at 0x7f6c50d2fb50>
Got ('This is the 1st broadcast',) {'kw': 'From the Observer'} From <__main__.Observable object at 0x7f6c50d2fb50>
Got ('This is the 2nd broadcast',) {'kw': 'From the Observer'} From <__main__.Observable object at 0x7f6c50d2fb50>

As you can see, the observable can directly interact with the observers and vice versa. The observable will be in interaction with the observer as long as the observer is subscribed to the observable's subscription list.

Pros & Cons

With the implementation in place, the pros and cons of this design pattern can be compared as follows:

Pros:

  • The one-to-many relationship is defined between the objects. This ensures that when an object is altered, it will lead to a cascade of changes to be applied to the dependent objects.

  • Loosely coupled objects mean that the components can be interchanged.

Cons:

  • The communication between the observable and observer is synchronous and with an increased load of subscribing and unsubscribing events, the observable object could be bombarded with requests. This could be mitigated by setting up a sleep time for each request.

  • The sleep solution could also cause a possible loss of speed, performance, and events. This was the main reason for the Pub/Sub pattern to have a message queue in between the publisher and subscriber.

  • Memory leaks are common in this pattern since there is a strong reference between the observer and observable. The observables need to be mandatorily deregistered from the observable object.

To mitigate most of the cons, a message queue was introduced in between the observer and the observable to overcome all these problems, which led to devising the Pub/Sub pattern - a variation of the Observer Pattern.

Conclusion

This guide covered the the Observer Pattern, how it can be implemented, and compares its pros and cons.

It's interesting to note that the Observer Pattern is one of the Behavioral Patterns that has led to many of the features we use today, such as RSS feeds, social media feeds, etc.

By being introduced to the nuances of the Design Patterns, it is easier to build the functionality from the ground up. And of course, knowing different Design Patterns allows you to build the best solution for different types of problems.



from Planet Python
via read more

How To Calculate Stocks Support And Resistance Using Clustering

How To Calculate Stocks Support And Resistance Using Clustering

In this notebook, I will show you how to calculate Stocks Support and Resistance using different clustering techniques.

Stock Data - I have stocks data in mongo DB. You can also get this data from Yahoo Finance for free.

MongoDB Python Setup
In [1]:
import pymongo
from pymongo import MongoClient
client_remote = MongoClient('mongodb://localhost:27017')
db_remote = client_remote['stocktdb']
collection_remote = db_remote.stock_data
Get Stock Data From MongoDB

I will do this analysis using last 60 days of Google data.

In [2]:
mobj = collection_remote.find({'ticker':'GOOGL'}).sort([('_id',pymongo.DESCENDING)]).limit(60)
Prepare the Data for Data Analysis

I will be using Pandas and Numpy for the data manipulation. Let us first get the data from Mongo Cursor object to Python list.

In [3]:
prices = []
for doc in mobj:
    prices.append(doc['high'])
Stocks Support and Resistance Using K-Means Clustering
In [4]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import AgglomerativeClustering

For K means clustering, we need to get the data in

(continued...)

from Planet SciPy
read more

How to Kick Off a Machine Learning Project With Less Data

If machine learning solutions were cars, their fuel would be data. Simply viewed, ML models are statistical equations that need values and variables to operate, and the data is the biggest contributor to ML success. Today, sources of data are ample, and the amount of available data keeps growing exponentially. This allows us to wrangle, […]

The post How to Kick Off a Machine Learning Project With Less Data appeared first on neptune.ai.



from Planet SciPy
read more

Tuesday, July 27, 2021

John Ludhi/nbshare.io: How To Calculate Stocks Support And Resistance Using Clustering

How To Calculate Stocks Support And Resistance Using Clustering

In this notebook, I will show you how to calculate Stocks Support and Resistance using different clustering techniques.

Stock Data - I have stocks data in mongo DB. You can also get this data from Yahoo Finance for free.

MongoDB Python Setup

In [1]:
import pymongo
from pymongo import MongoClient
client_remote = MongoClient('mongodb://localhost:27017')
db_remote = client_remote['stocktwits']
collection_remote = db_remote.eod_stock_data

Get Stock Data From MongoDB

I will do this analysis using last 60 days of Google data.

In [2]:
mobj = collection_remote.find({'ticker':'GOOGL'}).sort([('_id',pymongo.DESCENDING)]).limit(60)

Prepare the Data for Data Analysis

I will be using Pandas and Numpy for the data manipulation. Let us first get the data from Mongo Cursor object to Python list.

In [3]:
prices = []
for doc in mobj:
    prices.append(doc['high'])

Stocks Support and Resistance Using K-Means Clustering

In [4]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import AgglomerativeClustering

For K means clustering, we need to get the data in to Numpy array format.

In [5]:
X = np.array(prices)

For K means clustering, K which means number of clusters is very important. We can find the optimal K using the Knee plot as shown below.

In [6]:
from sklearn.cluster import KMeans
import numpy as np
from kneed import KneeLocator
    
sum_of_sq_distances = []
K = range(1,10)
for k in K:
    km = KMeans(n_clusters=k)
    km = km.fit(X.reshape(-1,1))
    sum_of_sq_distances.append(km.inertia_)
kn = KneeLocator(K, sum_of_sq_distances,S=1.0, curve="convex", direction="decreasing")
kn.plot_knee()

Let us check the value of K using kn.knee

In [7]:
kn.knee
Out[7]:
3
In [8]:
kmeans = KMeans(n_clusters= kn.knee).fit(X.reshape(-1,1))
c = kmeans.predict(X.reshape(-1,1))
min_and_max = []
for i in range(kn.knee):
    min_and_max.append([-np.inf,np.inf])
for i in range(len(X)):
    cluster = c[i]
    if X[i] > min_and_max[cluster][0]:
        min_and_max[cluster][0] = X[i]
    if X[i] < min_and_max[cluster][1]:
        min_and_max[cluster][1] = X[i]

Let us check the min and max values of our clusters.

In [9]:
min_and_max
Out[9]:
[[2461.9099, 2365.55], [2687.98, 2508.0801], [2357.02, 2239.4399]]

There are 3 clusters shown above, every cluster has max and min value.

At the writing of this notebook, Google stock price is 2687.98 (high of day) which happens to be the 52 high as well. Therefore based on the above clusters, we can say that 2687.98 is the resistance and next support level is 2508.0801. The next levels of support are 2461.9099, 2365.55 2357.02, 2239.4399.

Remember these support and resistances will change depending upon the range of data and value of Clustering parameter K.

Stocks Support and Resistance Using Agglomerative Clustering

In [10]:
mobj = collection_remote.find({'ticker':'GOOGL'}).sort([('_id',pymongo.DESCENDING)]).limit(60)
prices = []
for doc in mobj:
    prices.append(doc['high'])

Another approach that can be used is Agglomerative Clustering which is hierarchical clustering.

Agglomerative clustering is a bottoms up approach which merges child clusters to find out the big clusters of data.

I have found Aggloerative to be useful on stocks rolling data.

Let us create a rolling data of 20 days each for both calculating max and min values.

In [11]:
df = pd.DataFrame(prices)
max = df.rolling(20).max()
max.rename(columns={0: "price"},inplace=True)
min = df.rolling(20).min()
min.rename(columns={0: "price"},inplace=True)

Below step is required to prepare the data in two column format.

In [12]:
maxdf = pd.concat([max,pd.Series(np.zeros(len(max))+1)],axis = 1)
mindf = pd.concat([min,pd.Series(np.zeros(len(min))+-1)],axis = 1)    
maxdf.drop_duplicates('price',inplace = True)
mindf.drop_duplicates('price',inplace = True)

Let us use n_clusters =3 value for our number of clusters.

In [13]:
F = maxdf.append(mindf).sort_index()
F = F[ F[0] != F[0].shift() ].dropna()

# Create [x,y] array where y is always 1
X = np.concatenate((F.price.values.reshape(-1,1),
                    (np.zeros(len(F))+1).reshape(-1,1)), axis = 1 )

cluster = AgglomerativeClustering(n_clusters=3,
          affinity='euclidean', linkage='ward')
cluster.fit_predict(X)
F['clusters'] = cluster.labels_

F2 = F.loc[F.groupby('clusters')['price'].idxmax()]

# Plotit
fig, axis = plt.subplots()
for row in F2.itertuples():

    axis.axhline( y = row.price, 
            color = 'green', ls = 'dashed' )

axis.plot( F.index.values, F.price.values )
plt.show()

Let us plot our clusters now. As shown below, there are 2 clusters found. If we take in to account the todays closing price of Google which is 2638.00, we can say that 2687.98 is the resistance and 2357.02 is the support.

In [14]:
F2
Out[14]:
price 0 clusters
29 2561.1499 1.0 0
19 2687.9800 1.0 1
19 2445.1399 -1.0 0
37 2357.0200 -1.0 2

One thing to notice here is that, there are only 2 clusters at price 2357.02 which is not that many. To see if we can find more number of clusters either we have to increase our number of price points in our source data or increase the number of clusters, or make our rolling window smaller.

Let us increase the number of clusters to 5 and see what happens.

In [15]:
F = maxdf.append(mindf).sort_index()
F = F[ F[0] != F[0].shift() ].dropna()

# Create [x,y] array where y is always 1
X = np.concatenate((F.price.values.reshape(-1,1),
                    (np.zeros(len(F))+1).reshape(-1,1)), axis = 1 )

cluster = AgglomerativeClustering(n_clusters=5,
          affinity='euclidean', linkage='ward')
cluster.fit_predict(X)
F['clusters'] = cluster.labels_

F2 = F.loc[F.groupby('clusters')['price'].idxmax()]

# Plotit
fig, axis = plt.subplots()
for row in F2.itertuples():

    axis.axhline( y = row.price, 
            color = 'green', ls = 'dashed' )

axis.plot( F.index.values, F.price.values )
plt.show()
In [16]:
F2
Out[16]:
price 0 clusters
36 2399.0300 -1.0 0
36 2461.9099 1.0 0
29 2561.1499 1.0 1
37 2357.0200 -1.0 2
19 2687.9800 1.0 3
19 2445.1399 -1.0 0
51 2239.4399 -1.0 4
51 2436.0000 1.0 0

Ok this time around we got more number of clusters at price 2239.43 which is quite far from todays closing price of 2638. However the resistance number looks good of 2687.98 based on 3 clusters.

Let us make our rolling window smaller. Instead of 20 days let us make it 10 days.

In [17]:
df = pd.DataFrame(prices)
max = df.rolling(10).max()
max.rename(columns={0: "price"},inplace=True)
min = df.rolling(10).min()
min.rename(columns={0: "price"},inplace=True)
maxdf = pd.concat([max,pd.Series(np.zeros(len(max))+1)],axis = 1)
mindf = pd.concat([min,pd.Series(np.zeros(len(min))+-1)],axis = 1)    
maxdf.drop_duplicates('price',inplace = True)
mindf.drop_duplicates('price',inplace = True)
F = maxdf.append(mindf).sort_index()
F = F[ F[0] != F[0].shift() ].dropna()

# Create [x,y] array where y is always 1
X = np.concatenate((F.price.values.reshape(-1,1),
                    (np.zeros(len(F))+1).reshape(-1,1)), axis = 1 )

cluster = AgglomerativeClustering(n_clusters=5,
          affinity='euclidean', linkage='ward')
cluster.fit_predict(X)
F['clusters'] = cluster.labels_

F2 = F.loc[F.groupby('clusters')['price'].idxmax()]

# Plotit
fig, axis = plt.subplots()
for row in F2.itertuples():

    axis.axhline( y = row.price, 
            color = 'green', ls = 'dashed' )

axis.plot( F.index.values, F.price.values )
plt.show()
In [18]:
F2
Out[18]:
price 0 clusters
45 2318.8899 -1.0 0
45 2399.0300 1.0 4
19 2561.1499 1.0 1
26 2432.0601 -1.0 2
26 2461.9099 1.0 2
9 2687.9800 1.0 3
33 2412.8799 -1.0 4
33 2455.5100 1.0 2

Ok this data looks much better. We got a Google resistance around 2687.98 and support around 2399.03 and 2412.8799 which is quite close to say that support is around 2400.



from Planet Python
via read more

TestDriven.io: Working with Static and Media Files in Django

This article looks at how to work with static and media files in a Django project, locally and in production. from Planet Python via read...