Saturday, August 1, 2020

John Cook: Time spent on the moon

Lunar module and lunar rover, photo via NASA

This post will illustrate two things: the amount of time astronauts have spent on the moon, and how to process dates and times in Python.

I was curious how long each Apollo mission spent on the lunar surface, so I looked up the timelines for each mission from NASA. Here’s the timeline for Apollo 11, and you can find the timelines for the other missions by making the obvious change to the URL.

Here are the data on when each Apollo lunar module touched down and when it ascended.

    data = [
        ("Apollo 11", "1969-07-20 20:17:39", "1969-07-21 17:54:00"),
        ("Apollo 12", "1969-11-19 06:54:36", "1969-11-20 14:25:47"),
        ("Apollo 14", "1971-02-05 09:18:13", "1971-02-06 18:48:42"),
        ("Apollo 15", "1971-07-30 22:16:31", "1971-08-02 17:11:23"),
        ("Apollo 16", "1972-04-21 02:23:35", "1972-04-24 01:25:47"),
        ("Apollo 17", "1972-12-11 19:54:58", "1972-12-14 22:54:37"),
    ]

Here’s a first pass at a program to parse the dates and times above and report their differences.

    from datetime import datetime, timedelta

    def str_to_datetime(string):
        return datetime.strptime(string, "%Y-%m-%d %H:%M:%S")

    def diff(str1, str2):
        return str_to_datetime(str1) - str_to_datetime(str2)

    for (mission, touchdown, liftoff) in data:
        print(f"{mission} {diff(liftoff, touchdown)}")

This works, but the formatting is unsatisfying.

    Apollo 11 21:36:21
    Apollo 12 1 day, 7:31:11
    Apollo 14 1 day, 9:30:29
    Apollo 15 2 days, 18:54:52
    Apollo 16 2 days, 23:02:12
    Apollo 17 3 days, 2:59:39

It would be easier to scan the output if it were all in hours. So we rewrite our diff function as follows.

    def diff(str1, str2):
        delta = str_to_datetime(str1) - str_to_datetime(str2)
        hours = delta.total_seconds() / 3600
        return round(hours, 2)

Now the output is easier to read.

    Apollo 11 21.61
    Apollo 12 31.52
    Apollo 14 33.51
    Apollo 15 66.91
    Apollo 16 71.04
    Apollo 17 74.99

These durations fall into three clusters, corresponding to the Apollo mission types G, H, and J. Apollo 11 was the only G-type mission. Apollo 12, 13, and 14 were H-type, intended to demonstrate a precise landing and explore the lunar surface. (Apollo 13 had to loop around the moon without landing.) The J-type missions were more extensive scientific missions. These missions included a lunar rover (“moon buggy”) to let the astronauts travel further from the landing site. There were no I-type missions; the objectives of the original I-type missions were merged into the J-type missions.

One note about the Python code: subtracting dates works unlike you’d expect, depending on your expectations. The difference between an earlier date and a later date is positive. You might expect that when speaking of dates informally. But if you think of the difference in dates as subtracting the number of seconds from some epoch to a that date, you’d expect the opposite sign.

Incidentally, UNIX systems store times as seconds since 1970-01-01 00:00:00. That means the first two lunar landings were at negative times and the last four were at positive times. More on UNIX time here.

Related posts



from Planet Python
via read more

Andriy Kornatskyy: Python ASGI CLI

Call ASGI Python application module from command line (without an application server), just like CURL. Interested? You need to install asgi-cli package: pip install asgi-cli Usage asgi-cli --help usage: asgi-cli [-h] [--version] [-X COMMAND] [-I] [-H HEADER] [-d DATA] [-b] [-n NUMBER] [-v] app [url] positional arguments: app an

from Planet Python
via read more

Kushal Das: Use DoH over Tor for your Qubes system

I was using my dns-tor-proxy tool in the AppVMs in my QubesOS system. But, at the same time I was trying to figure out how to make it the default DNS system for the whole Qubes.

ahf provided me a shell script showing how he is forwarding the DNS requests to a VPN interface. I modified the same so that all of standard DNS queries become DoH queries over the Tor network.

Setting up sys-firewall

In the following example, I am setting up the sys-firewall service VM. All other AppVMs connected to this VM as netvm will be use dns-tor-proxy without any modification.

Make sure that the template for sys-firewall has the latest Tor installed. You can get it from the official Tor repository.

Download (or build) the latest dns-tor-proxy 0.3.0 release, and put the file (as executable) in /rw/config/ directory.

Next, modify the /rw/config/rc.local file & add the following lines.

systemctl start tor
sh /rw/config/dns.sh
/rw/config/dns-tor-proxy --doh &

As you can see, we are executing another script at /rw/config/dns.sh, which has the following content. Remember to modify the DNS value to the right IP for your sys-firewall vm.


#!/bin/sh

QUBES_DNS_SERVERS="10.139.1.1 10.139.1.2"
DNS=10.137.0.x

# accept DNS requests from the other vms

iptables -I INPUT -i vif+ -p udp --dport 53 -j ACCEPT
iptables -I INPUT -i vif+ -p tcp --dport 53 -j ACCEPT

# Clean up our NAT firewall rules.
iptables --flush PR-QBS --table nat

# We take incoming traffic on TCP and UDP port 53 and forward to
# our DNS server.
for QUBES_DNS_SERVER in ${QUBES_DNS_SERVERS} ; do
    iptables --append PR-QBS --table nat --in-interface vif+ --protocol tcp --destination "${QUBES_DNS_SERVER}" --dport 53 --jump DNAT --to-destination "${DNS}":53
    iptables --append PR-QBS --table nat --in-interface vif+ --protocol udp --destination "${QUBES_DNS_SERVER}" --dport 53 --jump DNAT --to-destination "${DNS}":53
done

# Log *other* DNS service connections. This part is optional, but ensures that
# you can monitor if one of your VM's is making any traffic on port 53 with
# either TCP or UDP. If you want to log *every* DNS "connection", including the
# ones to QUBES_DNS_SERVERS, you can either move these commands up before the
# for-loop in this file or change the --apend option to be an --insert instead.
iptables --append PR-QBS --table nat --in-interface vif+ --protocol tcp --dport 53 --jump LOG --log-level 1 --log-prefix 'DNS Query: '
iptables --append PR-QBS --table nat --in-interface vif+ --protocol udp --dport 53 --jump LOG --log-level 1 --log-prefix 'DNS Query: '

Now, restart your sys-firewall vm. And you are all set for your DNS queries.



from Planet Python
via read more

Weekly Python StackOverflow Report: (ccxxxviii) stackoverflow python report

These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2020-08-01 12:19:04 GMT


  1. Calculate min and max value of a transition with index of first occurrence in pandas - [10/2]
  2. Confusion related to Python's `in` operator - [10/1]
  3. How to combine numeric columns in pandas dataframe with NaN? - [9/4]
  4. How can I automate slicing of a dataframe into batches so as to avoid MemoryError in python - [8/1]
  5. How to sort a group in a way that I get the largest number in the first row and smallest in the second and the second largest in the third and so on - [7/5]
  6. How can i accept and run user's code securely on my web app? - [7/2]
  7. How to scale sklearn's pipeline output for Autoencoders? - [7/1]
  8. infer_datetime_format with parse_date taking more time - [7/0]
  9. Replace a list of characters with indices in a string in python - [6/2]
  10. How to Programmatically detect whether a file is a Python script - [6/1]


from Planet Python
via read more

S. Lott: Why Python is not the programming language of the future -- a response

See https://towardsdatascience.com/why-python-is-not-the-programming-language-of-the-future-30ddc5339b66.

This is an interesting article with some important points. And. It has some points that I disagree with.

  • Speed. This is a narrow perspective. numpy and pandas are fast, dask is fast. A great many Python ecosystem packages are fast. This complaint seems to be unsupported by evidence.
  • Dynamic Scoping Rules. This actually isn't the problem. The problem is something about not being able to change containing scopes. First, I'm not sure changing nesting scopes is of any value at all. Second, the complaint ignores the global and nonlocal statements. The vague "leads to a lot of confusion" seems unsupported by any evidence. 
  • Lambdas. The distinction between expressions and statements isn't really a distinction in Python in general, only in  the bodies of lambdas. I'm not sure what the real problem is, since a lambda with statements seems like a syntactic nightmare better solved with an ordinary, named function.
  • Whitespace. Sigh. I've worked with many people who get the whitespace right but the {}'s wrong in C++. The code looks great but doesn't work. Python gets it right. The code looks great and works.
  • Mobile App Platform. See https://beeware.org
  • Runtime Errors. "coding error manifests itself at runtime" seems to be the problem. I'm not sure what this means, because lots of programming languages have run-time problems. Here's the quote: "This leads to poor performance, time consumption, and the need for a lot of tests. Like, a lot of tests." Performance? See above. Use numpy. Or Cuda. Time consumption? Not sure what this means. A lot of tests? Yes. Software requires tests. I'm not sure that a compiled language like Rust, Go, or Julia require fewer tests. Indeed, I think the testing is essentially equivalent.
I'm interested in ways Python could be better. 


from Planet Python
via read more

Janusworx: A Hundred Days of Code, Day 023, Day 24 - Tiny Utility to do comparative DNS Lookups

Problem - Compare domain lookups, against DoH Servers

  • Take a list of domains (one per line) from a text file as input, find the IP address for the domain using the standard system level DNS, and then check against DoH answers from both cloudflare and google. and say if all answers match properly or not.

    Read more… (1 min remaining to read)



from Planet Python
via read more

Tryton News: Newsletter August 2020

This month we kept refining existing features to improve the user experience, smooth workflows and empower users.

Contents:

Changes for the User

The subject of the marketing automation and notification emails can now be rendered using the value of the record. This way the subject can be more specific which increases the chances of it being opened.

We added a view that can be opened from the product that shows the incoming and outgoing stock moves by warehouse. Each move has cumulative quantity fields which help the user adjust their plans in order to avoid shortages.

The German federal ministry of finance announced on June 30 a cut in value-added-tax rates from 19% to 16% for the standard rate and from 7% to 5% for the reduced rate. As the tax cut is temporary, lasting from July 1 until December 31, we introduced new tax definitions into the account_de_skr03 module for the periods before, during and after the cut.

All the wizard messages have been reviewed and normalized. Now they use proper icons and all the unnecessary exclamation marks have been removed.

The default color for the graphs is now managed by the client instead of the server. So the client can be customized to use a color which fits nicely with the theme.

The party module has been updated to use the latest version of python-stdnum and so includes more tax identifiers for countries like Andorra, Guatemala, Japan, Moldova, New Zealand, Peru, Paraguay, Uruguay, Venezuela and South Africa.
If you are missing a standard identifier, we’d like to remind you that the best way to get it in Tryton, it is to contribute it to python-stdnum.

We keep the last average cost price for FIFO products when the quantity in stock is below zero.

We automatically remove any carriage returns from the fields in the FEC export as they are not allowed.

Any closed accounts are no longer shown when opening the details of the balance sheet.

The clients place copied records at the same position as newly created records. This means that the clients inspect the order applied to the list and deduce the correct place for the new record.

The stock move assignation wizards have been reworked and merged into a single wizard. This now provides four options if all the moves cannot be assigned: “Cancel” to restore the the moves to their initial state, “Wait” to leave the moves partially assigned, “Ignore” to set the quantity to 0 for non assigned moves and “Force” to force the assignation of all moves.

You no longer need to set analytic accounts on the move that balances the non-deferral account when closing the fiscal year.

The reconciliation wizard now keeps you on the same account/party after each reconciliation if there are still more lines to reconcile.

Changes for the Developer

We require the cost price to be filled in for all outgoing and incoming moves now. It was already filled in automatically by the code but this is now enforced to ensure the developer doesn’t forget it.
We also require a unit price for drop shipment and customer return moves.

We’ve added model, record and records attributes to the wizards these are filled in using values from the active context. This simplifies and normalizes writing wizards.
Tryton also checks that the user has read access to the records before executing the wizard.

We have added a test to ensure it is possible to try and assign a move with unsaved values. This behavior is interesting because it allows you to set the value on the move as an assignation parameter while keeping the original values unchanged for the remaining quantities.

In order to minimize the data sent to the client, we now send only explicitly declared fields from actions instead of just excluding some. This is because we have found, over time, when a new field is added to the actions, we often forget to exclude it if it is not needed by the clients.

A common pattern used in Tryton to create grouped records is to use a list of tuples composed of key-value pairs. In order to use the itertools.groupby we need to sort the list. But sometimes it includes None values and in Python 3 it is not possible to order lists containing None. So we’ve added a tool sortable_values which takes care of this by providing a key function which can be used to order the list of values that may contain None.

It is now possible to define a keyword action that applies to any model. This is useful when creating a generic action which uses the active_model value.

We added an option to sendmail to raise an exception if it fails. With this it is possible to tie a transaction’s success to the sending of an email.

To the tools we’ve added a function to escape wildcards from strings.

It is now possible to define default values on routes. This can be used, for example, to set a default database on a simple route.

The session reset now uses an autocommit transaction which avoids an annoying concurrency error.

We’ve added an option to run cron just once. With this option trytond-cron runs the pending tasks and stops. This is useful if you want to use an existing scheduler like the UNIX cron or the kubernetes CronJob to reduce the permanent load of a running service.

We’ve converted Invoice.update_taxes into a dualmethod. This method is often used when creating a single invoice so this simplifies the code.

1 post - 1 participant

Read full topic



from Planet Python
via read more

TestDriven.io: Working with Static and Media Files in Django

This article looks at how to work with static and media files in a Django project, locally and in production. from Planet Python via read...