Python is a widely-used, interpreted, object-oriented, and high-level programming language with dynamic semantics, used for general-purpose programming. It was created by Guido van Rossum, and first released on February 20, 1991.
The most basic exercise when learning any new programming language is printing the phrase “Hello World”
Variables in python are different from most other languages. There is no need to declare the type of a variable explicitly; instead, python infers the type of the variable based on the value you assign to it.
For Example: The line
word = "bus"
will initiate the variable
word as a String because it uses quotations.
Python will do the same thing for all other supported datatypes including
We will only cover number, strings, and lists in this tutorial
Numbers include negative and positive integers and decimals All of the following lines are valid python code
num = 2 dec = 3.2 bignum = 32030492
Note: Python actually divides number into multiple datatypes but does not reveal the actual type to the programmer. You can see the actual type of the number using the type() method.
Python supports all the common arithmetic operations you would expect
x = x + 2 can be abbreviated as
x += 2
String are any character or group of characters which are initialized with quotations. Example:
word = "hello" a = "1AsdFsa" b = "#[email protected]"
Strings can be combined in the following manner.
a = "Hello" b = "World" c = a + " " + b # c is set equal to "Hello World"
Strings in python can be divided using a special “slicing” syntax
a[n] # gets nth character in string a[start:end] # string start through end-1 a[start:] # string start through the rest of the string a[:end] # string from the beginning through end-1 a[:] # a copy of the whole string
a = "You won't believe it's not butter." print(a[:3]) # prints "You" print(a[2:6]) # prints "u wo" print(a[4:]) # prints "won't believe it's not butter."
Lists are one of the most versatile python types and have a huge number of uses.
A list is a collection of other variables. Lists in python can hold any number of variables and are not dependent on the type of the variables that they are holding.
nums = [1,3,2] strs = ["hi","hello","hola"] mix = ["one",2,"three"]
You can use .append(element) to add elements to a list. .remove(element) removes the first occurrence of an element from a list Example:
a = [1,2,3,4] a.append(5) print(a) # prints [1,2,3,4,5] a.remove(2) print(a) # print [1,3,4,5]
The syntax for slicing used for strings can also be applied to lists.
a = [1,2,3,4,5] b = a[1:3] print(b) # prints [2,3]
For loops are one of the most important concepts in python. Any code put inside of a for loop will be repeated however many times its parameters dictate.
# Syntax for the for loop for i in range(num): # code to be looped over # num is the number of times the loop will be executed
for i in range(5): print(i) # The above prints 0 1 2 3 4
The enhanced for loop can be used when one wants to iterate directly over the contents of a list.
list = ["a","b","c"] for x in list: print(x) # The above prints a b c
If statements allow you to add logic to your programs. The if statement in python is composed of a condition and the code which will run.
x = 2 if x == 2: # Do something else: # Do something else # The code above will check if the value of x is 2 and evaluate different pieces of code depending on the truthfulness of the condition
There are several truth operators in python which allow you to evaluate statements
a == b # equals a > b # a is greater than b a < b # a is less than b a >= b # a is greater than or equal to b a <= b # a is less than or equal to b
Read about the input() function here: https://www.programiz.com/python-programming/methods/built-in/input
Try building a 4 function calculator with user input.
Create your own text-based story, where you can choose your own adventure!
To do this, you will need user input, print statements, and if statements. You may want to use a method or a loop if you’re interested in having the user playing the game again.
Today we will be using the online IDE repl.it to experiment with several python modules and build a web scraper for stocks.
What is a “module”?
A module is code that someone else has written that can be imported into a python program to be used.
For example, instead of creating a program to represent the date and time you can use the date time module in python
To use a module the
import statement is required
To show the use of modules we will build a stock tracker which scrapes the NASDAQ and will save data on apple stocks for us. This is useful for algotrading which uses algorithms to predict the future prices of stocks.
We will scrape the price of Apple (AAPL) stocks from https://www.nasdaq.com/symbol/aapl
To begin the program we will import all of the modules needed for our program. The first 2 modules we will need are BeautifulSoup4 which allows us to easily parse a webpage into an understandable format, and requests which allows us to receive the raw HTML of the webpage. We can import these modules as follows.
from bs4 import BeautifulSoup import requests
BeautifulSoup is part of a larger module called bs4 which is why we use the
We will also need the time and date time packages to record the times of the prices we scrape.
from datetime import datetime import time
Next we want to set up a dictionary object that stores a key which will be the time, and a value which will be the price of the stock at that time.
timeseries = dict()
We need an infinite loop to continuously collect datapoint. We only want to collect the price of the stock every 5 seconds so we will use the time module to force the program to wait for 5 seconds on each iteration of the loop.
while True: #More code will go here time.sleep(5)
To get the raw HTML for the page we use the requests module.
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") time.sleep(5)
Next we want to create a BeautifulSoup object which allows us to easily scrape the price of the stock
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") soup = BeautifulSoup(page.text,'html.parser') time.sleep(5)
Now we extract the price from the
soup object. The class parameter is the HTML class that contains the price on the website. You can identify the name of the class by using inspect element on the webpage.
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") soup = BeautifulSoup(page.text,'html.parser') price = soup.find(class_='qwidget-dollar').text[1:] time.sleep(5)
To add an entry to our dictionary we access the dictionary with square brackets and put the key of the new entry in the square brackets. Additionally we use the date time module to get the current time. The str() method converts the date into a string and the float() method converts the price, which is a string, into a number.
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") soup = BeautifulSoup(page.text,'html.parser') price = soup.find(class_='qwidget-dollar').text[1:] timeseries[str(datetime.now())] = float(price) time.sleep(5)
Finally we want to save our dictionary into a text file so that we can view the entirety of our results. The code to save data into a file is a little bit more complicated than things we have seen in the past so for now you can use the code provided. Make sure to create a file in the same folder as your python code called “file.txt”.
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") soup = BeautifulSoup(page.text,'html.parser') price = soup.find(class_='qwidget-dollar').text[1:] timeseries[str(datetime.now())] = float(price) with open("file.txt", "w") as text_file: text_file.write(str(timeseries)) time.sleep(5)
Optionally, we can use a print statement to view the results as our script collects stock prices.
while True: page = requests.get("https://www.nasdaq.com/symbol/aapl/real-time") soup = BeautifulSoup(page.text,'html.parser') price = soup.find(class_='qwidget-dollar').text[1:] timeseries[str(datetime.now())] = float(price) with open("file.txt", "w") as text_file: text_file.write(str(timeseries)) print(str(datetime.now()) + " " + price) time.sleep(5)
So we have seen that python modules allow us to build practical programs extremely easily. There are thousands of python programs available for you to try. See if you can think of other uses for BeautifulSoup or any other modules you find online.
Visit https://www.dataquest.io/blog/web-scraping-tutorial-python/ and learn more about how to use the module BeautifulSoup4. You can use the same python project as last week for these challenges.
1.Hard Go to IMDb and scrape a list of 20 movies with their metascore. Then sort the list from best score to worst score and print the list.