Installing Python, pip, Pyglet and OpenGL

I made the slight mistake of assuming that everyone would have Python and the various libraries installed before starting - or at least know how to get them, completely forgetting of course that I had no idea how to do any of this either!

So firstly go to the Python site, download and install python. I would recommend getting Python3 if you have the choice (although the difference won't make much difference for these tutorials). Just be aware if you're planning on using lots of old libraries or Pygame that you will likely want Python2 instead. Run the installer and, if you see the option, choose to update the system Path variable. (If not don't worry we will solve that later).

Next we're going to get python and pip running from the command line. Why? It will make it far easier to install any other external libraries you might want to use in the future - such as matplotlib, numpy, pyglet, ipython or sympy (all of which I'll use a lot in other tutorials!).  So open a command line (or terminal if you're on a Unix system), type 'python', hit enter and see what happens! If you're on windows, to launch the command prompt, click the windows icon on the bottom left corner of the screen, then type 'cmd' in the search box and hit enter. If you're on OSX go to launchpad in the doc -> Other/Utilities -> Terminal. If you're on linux i'll assume you can either do it already or can google it :p

If the command prompt has changed from a dollar symbol to these: >>> then congratulations! Skip the next bit and go straight to the Installing Pyglet section below (type exit() and hit enter to get rid of the >>> symbols and go back to the normal $ prompt). If instead you got 'python is not a recognised command' then follow the instructions below. (it just means that whilst you have python installed your computer doesn't know where to run it from - we can fix that!)

Windows

Open an new windows explorer window, navigate to your desktop, right click on 'My Computer', choose 'properties', then Advanced System Settings on the left hand side. The click the Environmental Variables button, and from the bottom list scroll down until you find a line called 'Path'. Select it and click 'edit'. Then add "; C:\Python34\Scripts; C:\Python34" without the quotation marks and including the starting semicolon, to the end of the text that's already. Hit save, re-start your command prompt and you're good to go! Alternatively, you can just replace 'pip' in the commands below with:

C:\Python34\Scripts\pip

UPDATE: If you're using python2.7 the the commands above should use Python27 instead of Python34

Other OSs

Google how to change your Path variable. Then add /usr/local/bin/pip and /usr/local/bin to it. Alternatively, you can just replace 'pip' in the commands below with: /usr/local/bin/pip

Installing Pyglet

So now that we have pip installed we can now install a bunch of useful libraries by opening a command prompt and typing: pip install <package_name> i.e. pip install numpy

We're going to install pyglet for Python 3 by using the command:

pip install --upgrade http://pyglet.googlecode.com/archive/tip.zip

The reason the command is so long is that we're using a 'development' bit of code (i.e. code that hasn't been exhaustively tested yet). If we were in Python2.7 then we could just type pip install pyglet  but sadly that won't work here! Now wait a little while while the library downloads and installs for you and then you will be all good to go. 

Final note while the package installs

Think about what you've just done. If you successfully updated your path variable you have just taught your computer how to find and run a particular program from the command prompt. That will work the same for any executable program from your computer - you can run them all from the command line by either changing your Path variable or using the full program location. This turns out to be super useful if/when you end up using the command line a lot. And I say that knowing this is most people's reaction to using the command line:

References

http://www.pyglet.org/download.html

http://www.reactiongifs.com/no-no-no-no/

http://stackoverflow.com/questions/17271319/installing-pip-on-mac-os-x

Pyglet - moving, rotating and translating images in 2D

Continuing on from the previous tutorial, now we can dig into moving things around in 2D. Why bother learning this when I've covered the same thing in Tkinter tut and when I'm going to cover it again in a pygame tut? Because in pyglet, the same commands for movement and rotation in 2D ALSO APPLY TO 3D, so once we make the jump to 3D we will automatically have access to all the colours, motion and movement actions we're already learned rather than having to grapple with all of them together at once. *

Translation

The translation command takes an object you've created and then moves it by a certain amount. In the previous tut, we controlled the shape and size of our triangle by where we placed it, but imagine if we wanted to move it after we had created it. Then we would have to calculate the location of all the vertices of the object every time it moved. So instead, we create an image close to our origin and use that to control its size and shape and then we move it to where we want in a separate command. This comes in really handy in 3D when figuring out where things are in 3D space and describing that in detail does my head in - so i cheat and use this command instead:

glTranslatef()

The three numbers we pass the method tell OpenGL how far and in what direction we want to move our shape in the order x,y,z. Some helpful tips to remember:

• x = left or right motion. Negative numbers move left, positive moves right.
• y = up or down motion. Negative numbers move down, positive moves up.
• z = motion into or out of the screen (3D only), Negative numbers move into the screen, positive moves out of it.

And yes, because we're in 2D all our transformations currently will leave z =0.0. Try changing it if you like and see what happens, but it's likely your image will disappear because we haven't quite set our system up to display 3D well yet.

Rotation

Rotation is similar to translation except it requires some serious math. So once again, rather than bothering with the details we use an OpenGL command to do all the heavy thinking for us. This time though we have 4 input options:

glRotatef(10.0,0.0,0.0,1.0)

They're:

1. the number of degrees to move
2. 1.0 if rotating around the x axis, 0.0 if not
3. 1.0 if rotating around the y axis, 0.0 if not
4. 1.0 if rotating around the z axis, 0.0 if not

So a full program making use of both translation and rotation would looks something like:

Note: We've also included 2 other commands 

    glMatrixMode(GL_MODELVIEW)

    glLoadIdentity()

These aren't explicitly necessary - take them out and everything will still work fine. However, once we move into more complicated motion these will become really important, because we're going to have to change between viewpoints and these are the command that 'bring us home' (i.e. back to where we started). For those that are interested, OpenGL works by using a matrix to transform/move/rotate our objects depending of our starting position. If we move to a new location then the same matrix will have a different effect - so we 'come home' by loading the identity matrix in case we've accidentally changed location without realising it. Yes this is why linear algebra is important for computer graphics. No it's not the only reason - but it's one of the prettiest. :) 

Challenge!

Does the order of the translation and the rotation commands matter? Is that weird? Can you figure out why or why not? What happens to your image if you rotate it around the x or y axis? Why?

Movement

Time for another method that I mentioned earlier - update().

def update(dt):

    global rz

    rz += dt * 10

    rz %= 360

ry=0

pyglet.clock.schedule(update)

The update() command every time pyglet's internal clock 'ticks', so if we change the position just a little with every 'tick' as long as it happens fast enough this will look like movement. Here's we're probably better to explain by example:

The details are - we create a global variable called rz (the rotation around the z direction)

Challenge!
What happens if you set rz to be constant??? Why? What would you need to add to get translation working?

*I'm certain this can also be achieved in pygame with or without external libraries (see examples here) and if I ever learn how with that I'll post a tutorial on it.

Intro to pyglet and OpenGL

After taking a look at this awesome Minecraft demo written in Python, I decided to do a little digging and playing with the library it uses to use the OpenGL graphics renderer with the ultimate goal of making a 3D game from scratch as an outreach project. Predictably 'having a play' took quite a while to get working because I'd never seen much OpenGL before so I thought I would put together a step by step guide to getting started for those NOT familiar with OpenGL but with some experience of Python.

Here's a minimum working example of a pyglet window using the openGL commands to draw something simple:

Looking at this step by step, we're importing our library as per usual, creating a window, defining an method called 'on_draw' and then initialising the entire program using the line: pyglet.app.run()

You may notice that the one method we have defined isn't actually called anywhere - that's because it's called automatically within the pyglet.app.run() method. There are other functions that are automatically called as well; update() and on_resize() which we will come back to - but they're there to make things easier for the programmer. They trigger at particular time intervals and when the window is resized (or created) respectively.

Looking at the OpenGL commands, we first declare what shape of object we're going to draw with glBegin(GL_TRIANGLES) (we could also pick GL_POINTS, GL_LINES, GL_QUADS and more)
then we specify the location of the vertices, and then we declare the glEnd() so our program knows we're finished drawing and can start rendering for us.

So what's a vertex? It's a point or a corner. If you're drawing points then the 'vertex' is just the location of the point. If you're drawing more complicated shapes then the vertices are the locations of the corners (so a triangle will have 3, a quad [a shape with 4 sides] will have 4 and a polygon can have as many as you want). Pyglet then 'fills' in the rest of the shape for you by filling in the locations between the corners. So what's with the '2i' bit? That's how many dimensions your drawing in. In this tutorial (for simplicity's sake) we will stick to making and moving things in 2D but the next tut will show you how to generalise into 3D.

Challenge!
Using only what I've used above, a couple of other commands below, the power of google (and educated guesses!) try making something impressive out of ONLY the simple shapes described above.
One point to know is that the location (0,0) corresponds to the lower left-hand corner of your window, hence if your window is 400 x 400 pixels, the then upper-right corner is going to be at location (400,400), the upper-left at (0,400) and the lower-right at (400, 0).

Colours
Everything is more impressive with colours, and our image is sorely lacking them. Adding colours in is simple - just add in the commands, and play with the numbers to change the colours:

That said, if you have overlapping triangles then colouring can be a real pain - so if you want to just display outlines of your triangles (to start with anyway) just put in this command and then comment it out when you don't need it.



Challenge!
What happens if you give different corners of the same triangle different colours?

Shortcuts
Ok, defining every point by trial and error quickly gets really tiresome. Conveniently there's a few shortcut commands to speed up the process thanks to pyglet:

This command cuts out the need for the glBegin and glEnd and the individual vertex commands and just gives pyglet a list of locations to connect into a triangle. Here's a working example of its use:
Methods
To do really cool stuff with pyglet (and programming in general!) we're going to have to wrap our code up into nice little chunks called 'methods' which we can call again and again rather than copy-pasting our code all the time. The on_draw bit from the code above is a method that is run every time the program loops. We're going to define another one for making triangles:
Try including this in your code (Hint: put the method BEFORE the on_draw function and call the method INSIDE the on_draw function instead of the current code we have in there).

Extension Challenge!
Given your experience with these shapes now - what's the most complicated design/shape/image you can represent using only triangles? Here's a clue - the below image is called a Voronoi tessellation (google it - there's some amazing art out there based on this technique).

Next tutorial we will use the OpenGL rotation and motion tools in 2D

References:
http://www.pyglet.org/doc/api/pyglet.graphics-module.html
https://github.com/fogleman/Minecraft
http://stackoverflow.com/questions/7681899/moving-an-image-around-in-3d-space
http://stackoverflow.com/questions/4269079/mixing-2d-and-3d-in-opengl-using-pyglet

Sorting emails and spreadsheets with Google Apps Script

I get far too may emails that I can deal with on a daily basis so I use Gmail filters to help me keep track of what is where. However sometimes you want to extract information from these emails without having to read every single one - for instance if you wanted to keep a database of all the emails you received from a mailing list and their content. Google spreadsheets provides several nice tools to deal with this - most notably their Google Apps Script through which you can access much of the Google toolkit's functionality (i.e. sending emails, generating charts etc). Their scripting language is based on Javascript - so even if you're not familiar with it it's not too hard to pick up with some specific examples.

So diving right in - open a new google spreadsheet and from the file menu at the top select Tools -> Script Editor which will open a new window for you to enter your scripts in. The first thing to note is that this script is currently locked to the spreadsheet you opened it from so if you run the commands:


the script will load the spreadsheet you started from and then set the variable 'lastRow' to the row number of the last row with information in it.

If you want to know what values that variables has you can print it to the console log using the command:
(If you're coming to this from Python this is akin to the print command).
Finally, the commands you want to read from or write to a spreadsheet cell are:

To check everything is working, enter some numbers into the first column of your spreadsheet, then copy and paste the following script into your script editor and then select 'Run' from the file menu. You will have to give the app permission to run - and to see the log output choose Logs from the View menu (The execution transcript from the same menu can also be useful for debugging).

All our script is doing is reading through the spreadsheet, printing out the value it finds and then replacing it with the square of whatever it finds. Simple!....unless you accidentally put text in in which case the script will throw an error. I'll leave you to sort out error handling on your own with the Google Apps Script documentation.

So how do we use this to keep a log of information from emails? Like this:

Just remember - for this to work the emails must have the label 'data_emails' (unless you change the variable in the code) and must be marked as Unread.
Now you have that data in spreadsheet format - you can process/sort it with other scripts, send emails that contain it, or automatically graph it each day. I'll cover some of these in future tutorials

Scraping websites with Google spreadsheets

Frequently when working with data collection or visualizations it's useful to get data from an external website that doesn't have an API. There are plenty of tools to help you with this but one of the easiest is certainly google spreadsheets, or you could of course write your own web scraping bot with Python and the Beautiful Soup library (which I will outline in a future tutorial). One thing to note - web scraping is not always legal so make sure to check the website's data use policy before you go scraping it and sharing it with everyone!

To get started go grab a google account and/or login to google documents and create a new google spreadsheet. then go find the URL of the data you want to scrape. To keep things simple I'm going to demonstrate with the highest-grossing films data from Wikipedia. Here's the URL:
 
http://en.wikipedia.org/wiki/List_of_highest-grossing_films#Highest-grossing_films

If you're not familiar with URLs, the # character directs your requests to a specific subset of the page - in this case to the 'highest grossing films' section. If you go to the site you will see a table which is the data we're interested in:



In the top left hand cell of your spreadsheet, use the following command:
 =ImportHtml("http://en.wikipedia.org/wiki/List_of_highest-grossing_films#Highest-grossing_films","Table", 1)

Hit enter and the spreadsheet will slowly populate itself with all the data from the table we're looking at and you can graph it or save it as a .csv or text file for later analysis with another program or script.

If it didn't work there's a few things to check:

1. Make sure you have enough of the right kind of quotation marks. There should be double quotes around both the URL and the word Table in the above command
2. Check the URL is correct.

So what precisely is happening? The =ImportHtml() command directs google spreadsheets to the particular page and the next command "Table" tells it too look for an element in the format of a table. You can also use alternative Html elements here like lists ("List"). Finally the value 1 tells it which element to look at. Start with 0 and keep increasing the number by 1 until you find the data you're looking for. 

Have a go changing the number in the above command and see if you can scrape the table from further down the page with films adjusted for inflation.


There's a functional example of the spreadsheet here if you're interested.

It's also worth noting that Google spreadsheet come with several other import tools for importing different types of data including .xml with:

• ImportHtml()
• ImportXML()
• ImportData()
• ImportFeed()

Finally you can also use the other google spreadsheet commands to sort through the data you get. One of the most useful is the =SPLIT() command that separates the incoming data by a certain character (so the command =SPLIT(A1, " ") would split whatever text is in the cell A1 wherever there was a white space.)  Another useful command is =REGEXEXTRACT(), which searches a sting for a particular pattern of letters and numbers known as a regular expression, and then returns it. Regex commands are a little complicated to get used to but really useful - here's one that searches for the pattern of a year(i.e. 2014) in a string:  
=REGEXEXTRACT(A1,"[0-9]{4}"
)