Thursday, 11 December 2014

MIT App Inventor - Accelerometer App

At 2014 CS4HS myself and several of the physics teachers decided grabbing the accelerometer data from a smartphone could be quite a good way of linking physics with digital technologies, but of course it's more fun to get the students to write it themselves than to get me to do it for them!

Regardless here's the rough outline for w MWE (minimum working example). You will need 6 label elements (one title and one reading for each direction), the accelerometer element and a clock element. I arranged mine using the horizontal arrangement sections like this:
Then all we need to do is connect the blocks in such a way that  the accelerometer data is printing to the screen with each 'tick' of the clock. Doing it this way prevents the phone trying to query the accelerometer sensor too quickly.

Challenge: Extend your app to store the accelerometer data using the TinyDB element to be printed out to the screen (or possibly to a file for plotting) at a later date

Friday, 25 July 2014

Tkinter - Random Backgrounds with Python, version 2!

Our original random background generator was rife with errors so we've restructured it to make it a little better. See if you can follow the math and where I went wrong in the last one!

Tuesday, 15 July 2014

Python - Level 3 Material

Level 3 Python Topics


Tkinter

Tkinter is a python module for creating simple graphical user interfaces (windows and elements). Whilst I covered the basics of making a Tkinter canvas and drawing shapes here, I'll also mention how to add buttons and text fields etc. Here's something we might start off with:

Now to add a button:
But this button doesn't do anything. To attach some functionality to it (in this case we're going to overwrite the blue rectangle with a white one) we use the 'command='statement in the Button() method and then we write a method linked to that (in our case called doClear).
But we can also add more than buttons - for instance here's a text_field with an associated label:
And here's a dropdown menu that allows users to pick from a series of predetermined choices:

Challenge: 

Make a Tkinter GUI that allows the user to pick from a list of countries and then draws out their flags using the Tkinter canvas create_rectangle/arc/oval commands when the 'Draw' button is pushed. I suggest starting with Japan, Germany, Norway and Scotland. 

Object Oriented Programming

Object orientation is a way of re-organising your code in such a way that it makes doing lots of complicated things much easier (a bit like we did with methods in the Level 2 section). The only way I managed to wrap my head around it (and it took me ages!) is to think of an illustrative example. Imagine you were writing a simple game like pong for instance.

Essentially you need to create two paddles and a ball (ignoring scoring), remember where each is, what direction it's travelling in and at what speed, where its edges are etc. You can do this pretty easily with only 3 items (although it will require a fair few variables). But imagine what would happen if you tried to extend pong in any way; What if you decided to add 2 balls? Or 3? What if you just wanted to add in a third dimension? The number of variables and complexity gets huge! And we're just doing more of the same, so instead we write a Class for the ball and paddles and then we just say how many of each we want. Each class remembers its own set of variables, which means we only have to define each one once and every new copy already has its own variables. For instance the Ball class might be defined like this:
The full code of a ball class might look like this: and now if we want to add bouncing functionality off the walls (and generally clean up the code)

Challenge: 

First, extend the code above to have >100 balls all starting at different locations (this should only take about 5 more lines of code). Make Pong using Tkinter and a class for the GUI, paddles and ball(s).

Final Challenge:

Build a 'mosaic maker' program - a program that generates a grid of squares in a GUI canvas and allows the user to change their colour by clicking on them (example shown below). The GUI should have 2 buttons that allow the user to select either red or blue for the colour of the clicked square (the default should be black) and one that allows the user to clear the grid to white. Finally you may want to extend your program to allow the user to replace a square with a .png image of their choice or to choose particular colours by entering their numeric RGB values, and write a doctest that tests your programs ability to deal with non-RGB inputs. Finally re-structure your program so that you have a class for the GUI.

Python - Level 2 material

Level 2 Python Topics



Methods

In the Level 1 material we used a number of 'methods' already (like rantint() and range()). These were predefined by other people - but we can just as easily write our own to do what ever we want, is particular this comes in really handy when you want to do the same or similar things multiple times. Rather than copy pasting our code - we can just repeat a method with different inputs. 


Challenge: 

Construct a method that picks a random number between 2 numbers passed to the method but that DOESN'T include the numbers themselves.

Importing Text files


Importing text files and processing them is how I actually got into python, it's certainly some of the simplest functionality that allows you to do a huge amount!
As with everything we can also wrap it into a neat little method to use again later:

Lets say you're interested in global companies performance - you can get data from Wikipedia about their financial performance and turn it into a csv or txt file using Google spreadsheets (tutorial here), but in this just download this data as a csv.  

Navigating Lists

Now we've converted our text file to a list we can now navigate it's rows and columns easily


Challenge: 

Using this table from Wikipedia (and a Python program), determine what's the most common starting letter for the name of one of Snow White's 7 dwarves (or just the most common name). 
(N.B. you can also use python to automatically get the data from the net for you using a module called urllib2 and the method urlopen(url) but it's far easier to start with a text file.)

Sanitising User input

Here's a few nice tricks for sanitising user input - can you figure out what each line does?

Challenge: 

Wrap the above code into a method that takes a text question and a list of acceptable responses and returns the user's answer when they give an appropriate one.

Final Challenge

Create a program that counts all the individual letters in a book and how many times each one occurs, then prints the results (I suggest a copy of Charles Dickens' 'A Tale of Two Cities' which you can get free here). Why? You might be able to use the frequency of certain letters as a sort of 'fingerprint' for the language of that book (i.e. is it in Modern English, Old English, German, French or Latin?), but also this 'frequency analysis' is the most basic method of code cracking. Lets say you receive some encoded text that has been encoded with a simple substitution cypher (i.e. one letter has just been replaced with another). A frequency analysis like this will show (roughly) which letters are which in the encoded message allowing you to start to decode it (for example in English the most common letter is 'e' so the most common character in the encrypted message is probably also 'e'). If you want to test your cracking skills find a nice long string and encode it with the command 'encode':
Then do a frequency analysis on the encoded string and see if you can figure out what character has been replaced with what (ideally you would have someone else encode the message so that you didn't know what the original message was!).

Python - Level 1 material

I've been running introductory Python sessions with students and teachers from around Wellington for the last few months looking at the material relevant to NCEA level 1, 2 and 3 digital technologies achievement standards. In the interests of discussion and improvement I'm going to post what I cover here along with the Challenges I pose to participants along the way to get people to use the knowledge I'm allegedly imparting to them over the course of the session. So without further ado here's Elf short, sweet foray into Level 1 Python (I'll cover level 2 and 3 in other posts). Feel free to copy and paste this code into ipython or a python script and run it as you go so you get a feel of what each block does.

Level 1 Python Topics


  • User information (input())
  • String editing and concatenation 
  • For Loops (for i in range():)
  • Counting (i+=1)
  • Random numbers and modules (import)
  • Conditional logic (if:)
  • While Loops (while True:)

User Information and string editing


Challenge: 

Ask the user for their name, address, phone number and the colour of their house. Then print out a description in the form of: "Your name is <name> and you live in a <colour> house at <address> and I can call you on <phone_number>".

For Loops


Challenge:

Using a for loop within a for loop print the numbers 0-10 3 times.

Counting/Iterating


Challenge:

Ask the user for a number then count down from that number.

Random and modules


Challenge:

Ask the user for a number and then print that many random numbers between 10 and 100. Hint: you will need to convert your user unput from a string to an integer using the int() command.

Conditional Statements


Challenge:

Pick a random number. If less than 10, or if equal to 30, or if it's not greater than 50 print 'Yes'.
Otherwise print 'No'.

While Loops


Challenge:

Set a while loop that picks and prints a random number that has a 1/100 chance of breaking from the loop.

Final Challenge:

I find the best way to learn these concepts is to have a challenge that uses them all. A nice simple example is creating a computer game of Rock, paper, scissors which you should be able to make using only the coding elements I've described above. You will need to ask the user for their choice of rock, paper or scissors (can you allow the user to ONLY select from these somehow?) - then get the computer to pick as well (those random commands might come in handy) - then compare the two inputs and decide who wins and (as games usually run the best of 3) you will need to count each players score and decide which player has won. And once the game has completed you might want to ask your players if they want to play again? 

Finally - if that's all too easy for you I suggest creating an 'expansion' to your game that plays rock, paper, scissors, lizard, spock following this logic:

Monday, 14 July 2014

Blender 2.7 - Rigid Body simulation

Blender is great, but it's a little tricky to get anything really satisfying out of it quickly. That said, beautiful projects like QWOP or this video of collapsing barrels from the Crysis engine shows that simple designs can give us some really epic animations!

This tutorial was originally conceived by Andrew Price from Blender Guru, but we're going to expand on some of his ideas.

Ball and Chain

First off we're going to create a linked chain of rings. Open a new blender project, delete the default cube, and replace it with a torus. Rotate the torus 90 degrees from the default plane, then duplicate (Shift +D) and move the second ring to be threaded perpendicularly through the first. Duplicate these links until you're happy with the length of your chain. Be careful when placing them to ensure that each ring doesn't overlap. Then we just need to tell Blender that these are rigid body objects. Select one of the links - then head over to the Properties editor (right side of the screen), select the physics tab (the blue bouncing ball) and select "Rigid Body" and from the "Shape" field change the default from Convex Hull to Mesh (this just changes how the rigid body physics are calculated). We can easily apply this to all the links in the chain by now selecting all of them (Shift and Right click), and choosing 'Copy from Active' on the Physics tab on the left of the screen.  Now test out your animation skills by pressing Alt+A. If your chain falls down then you've done everything correctly. If it explodes (as mine did!) then check that you have changed the shape object to Mesh for each link in the chain. Add a plane and scale it up (S) to give your chain something to fall on. However, to stop the plane from just falling down you will need to set it up as a rigid body object (as we did for the chain) and simply change the type from Active to Passive. You can pin the chain at the top by selecting the top chain link and de-selecting the 'Dynamic' checkbox in the Rigid Body pane. And feel free to add a wreaking ball to the end of the chain by selecting the bottom link, going into edit mode (Tab) adding a UV sphere and overlapping the two. Now you have everything you need to make a swinging ball and chain (increase the mass of the links to increase the chain strength).

The array modifier

Now we want to generate a large number of things for our wrecking ball to slam into (or drop onto). For this we can use an array modifier (which is found on the right hand side panel in the tab with the little wrench). Create a default cube and apply the array modifier to it and as you increase the array number you will see the number of repeats increase as well. You con control the axis of the repeats by the sliders under the Relative Offset box. Add an array modifier for each axis (i.e. 3) with a number of 1.02 to each (allows a little gap between all boxes) and apply them to get a stack of boxes (this will work with any shape so go nuts with Monkey heads instead if you like). Finally, as we did above, apply an Active Rigid body element to each box, reduce the mass to 0.1 - then test with Alt+A! If you've done this correctly, your boxes should collapse and explode. What?!
We just need to correct the centre of mass for each box (currently it's all placed on the original box). We do this by selecting all the boxes, the CTRL+SHIFT+ALT+C and selecting "Origin to Geometry". Now run Alt+A again and adjust your simulation. Enjoy!

Challenge: Use your wrecking ball (or another means) to topple a tower of 30+ cylinders

References

The original reference for this idea on how to do it comes from Andrew Price's video available here:
http://www.blenderguru.com/tutorials/quick-tutorial-make-a-wrecking-ball-with-rigid-body-physics/#.U8RuZ0AW3J4. Andrew has also written a bunch of other fantastic tutorials which I highly recommend here: http://www.blenderguru.com/tutorials/

Sunday, 13 July 2014

MIT app inventor - Buttons, Text to Speech, Web and Translate

We just ran this as a workshop at the association of Women in Science conference, so we though we would put it up as an online tutorial too. This will take you through the basics of writing and MIT app inventor App, downloading and installing it to your phone and then following through a few extension challenges beyond those from the standard App inventor tutorials (with a healthy dose of elvish impishness!).

Head over to http://ai2.appinventor.mit.edu/ and login with your google account (or register for one for free) then take a look at the MIT App inventor basics tutorials here: http://goo.gl/sVZMzK.

It's worth noting that you DON'T need an Android phone to test these out as if you have installation privelages to the computers you're working on then you can simply install an emulator. Unfortunately this is only currently an option for OSX and Windows, which also allow direct connecting of an Android phone to a computer with the MIT App inventor editor on by Wifi or USB cable. This awesome little feature instantly clones the app you're creating to your device allowing you to test it in real time. If you're a dedicated linux user like us though, then a nice workaround is to simply use the 'Build' functionality at the top of the page to build your program and then automatically download and install it to your phone using a QR code (however you can end up with a lot of apps installed to your phone this way!).

Our first app: Text to Speech

Aim: An app that speaks some text that the use inputs
        Instructions: Here
Challenge: Extend your app so that it screams when you shake it (check out the .shaking method in the accelerometer sensor)


Our second app: Ball bounce


Aim: An app that allows you to control an on-screen ball
        Instructions: Here
Challenge: Increase the numbers of balls to 5, and allow collisions between balls

Repeater

Aim: An app that recognises voice commands and reads them back to you.

        Instructions: 

Start a new project and set up a basic screen with a button, a label, a speech recognizer and a text-to speech-element. Using the blocks editor, set up these blocks:
Simply, when the button is pushed, the app will try and recognise what has been spoken and will print a copy to the label and then attempt to speak it outloud. Yes, the SpeechRecognizer isn't great at NZ accents. Yes it's a source of much hilarity. Convieniently MIT app inventor have just introduced some new translation functionality in the form of the YandexTranslate design element (found under the 'Media' tab). You can pass this a string of words and it will attempt to translate it into a language of your choice.

Challenge: The 'repeater' app reads back to you in Italian


If you're looking for other languages:

  • en - English
  • es - Spanish
  • de - German
  • no - Norwegian
  • fr - French
  • ru - Russian (however the altered character set will appear in the translation label but not be read out by the text to speech element)

Insulter

Aim: An app that scrapes data from a website and prints it on screen

        Instructions: 

As per usual we're going to start off with a button and a label field, but because we also need data from the internet we will also need the 'Web' element from the Connectivity tab. The set up some blocks:
The first block connects the button with retrieving the html of the website. The second block triggers when that call is complete - and contains and 'if' statement that tests is the website retrieval was successful (responseCode = 200) or not. If it was successful then the returned html is printed to the screen and then spoken by a text to speech element. Initializing the local variable isn't necessary here - but becomes useful in the next step. And yes, you're reading this code right - this will just print the entire html of the website.

So our next step is to 'parse' the website html to allow us to pull out the elements we want. To see the html I'm talking about, head over to http://insult.dream40.org/, right click and select View Page Source. If you look though it you will quickly find two lists of words that the website picks from to get its insults.We can use a 'split' block from the text tab to help us sort through it. I suggest starting by getting your program to grab one insult from the website and display it - then modify that to allow it to grab multiple elements and choose randomly between them. There IS an inbuilt Html Parser method in the Web element, but we were unable to get it working so we just did it by hand!

Challenge: Gets insults from http://insult.dream40.org/ and speaks them.


If you're interested in doing more you can follow our MIT app inventor bluetooth tutorial here: http://goo.gl/QbQhLk

References:

We've borrowed lots of this material from the MIT app inventor team of course! you can find all their wonderful tutorials here: http://appinventor.mit.edu/explore/ai2/tutorials.html and they're by FAR the best way to learn all about App inventor.
We also found this tutorial about getting Yahoo Stock prices really helpful: http://beta.appinventor.mit.edu/learn/tutorials/stockquotes/stockquotes.html
For those that want access to more Android functionality - we suggest Android studio rather than the old Eclipse plugin.

Tuesday, 3 June 2014

Pyglet - Keymapping and Sound effects

For any real game however you probably want some conditional events - like controlling an object or causing something to happen when a button is pressed. In the below example we map three keys; a, left arrow and the left mouse button to 3 separate actions.


A couple of things to note:

  • to ensure the sound effect plays at the same time as the key is clicked we load the resource into memory instead of streaming the audio file as we did in the earlier example.
  • for keymapping the 'modifiers' field indicates where Caps, Shift or Ctrl is held.
  • Finally for mouse events and x and y is returned giving the location of the mouse cursor in the window when pressed.

Challenge!

Using this sounds effect, create a game that creates yellow ovals at the mirrored mouse location when the left button is pressed. I.e. if a mouse click was at (200, 100) in a 600x800 window then the 'mirrored' location should be at (600-200, 800-100).

Pyglet - placing images, text and playing background music

Continuing on our Pyglet introduction you can use the basic environment we set up earlier to place external images (in a few different formats), display text and to add some atmospheric background music. This code is pretty self explanatory so I'll just post the lot and let you dig through it yourself - but one thing to notice is that by itself pyglet/Python can't decode and play .mp3 audio files. It requires an external library called AVBin. So if you get an error when trying to play .mp3 files try installing the AVBin library (see the link below for details).


There are a few other commands in here that are worth pointing out:

  • image.blit(0,0) - the (0,0) is the location of the centre of the image  in x and y co-codinates. 'Blitting' is the process of drawing an image into our window. Otherwise we only load it into memory and never display it.
  • we also use the window.height and the window.width commands to return the current height/width of the window which is a far nicer way of defining the location of things. (N.B. the height and width methods also work on image objects!)

Challenge!

Find some nice atmospheric music for your game (and yes check it's Creative Commons licenced for you to use first!) and setup a background image you want to use and display it in the centre of your image in such a way that if you resize your window it appears in the the same place. I highly recommend checking out the wonderful local composer Rhian Sheehan's Soundcloud for inspiration (and yes you can download some of his .mp3s but make sure to attribute them if you use them!). 

References


Stellarium - Intro to basic scripting

I am a space nut. I love everything about space, from how weird and mysterious it is, to how vastly, hugely, mind-bogglingly big it is to Douglas Adams. It's difficult to explain at the best of times and something like a static powerpoint simply never does it justice. In my travels I have discovered quite a few programmes for exploring the universe, including the weighty Celestia, but my favourite in terms of ease of use is currently Stellarium. It's free and open source and a great way to get into exploring the sky in some detail. Also it comes with a scripting interface and a host of complex examples - so in this tut I'm going to describe the very basics of setting up and automating a Stellarium script.

Installation and Use

Download and install Stellarium here. Run the program and it will show you the stars from Paris for the current date. Have a play and get used to the controls (a full list is here), but some useful basics:
  • A - toggles the presence of the atmosphere
  • P - toggles the labels of the planets
  • Ctrl + Up/Down - zooms in and out
  • Space centres on the selected object
  • F12 - displays the command window (for scripting)
Expanding the bottom left corner pane reveals a spanner, select it and navigate across the the scripts tab to find all the loaded demo scripts. Finally, to edit or create new scripts you will find them in the Stellarium directory's 'scripts' folder and you should be able to edit them with any text editor (if you keep them here they will automatically appear in the scripts list).

Time and Place

The most confusing thing (for me!) about planetarium software is getting the time and place correct that you're looking to replicate. In Stellarium there are two commands for setting and getting time and position data:


These commands return the current time and location in Stellarium and set those as the current location. Nothing too magic here, but you can also pass the set commands any date or location and it will display what is visible from those coordinates.
i.e. These are the stars at 5am on the day I was born, as seen from Wellington, New Zealand:

Setting focus

Because of the aforementioned hugeness of space, it's inadvisable to to jetting off without knowing roughly where you want to go. For that reason, the easiest way to script is to select your target object, turn to face it and them move directly towards it. In Stellarium we can do that with the commands:


The second line of the above sets Stellarium to keep tracking the object as it moves through space. Then we can zoom in and out with:



with the core.wait(5) command setting the script to pause between commands, for 5 seconds (it's always a good idea to do this to allow the camera to complete one motion before starting another). We can direct the zoom to a specific scale with:


Challenge!

Generate a script that centres on and then zooms to the planets in order, then speeds up time to show the rotation of each planet, then finally zooms to our nearest star (Proxima Centauri) and looks back to see our sun from there.

As as per usual here's a full functional script that centres on the moon and then zooms to show Jupiter and the Galilean moons as an example.

Thursday, 29 May 2014

Installing Python, pip, Pyglet and OpenGL

I made the slight mistake of assuming that everyone would have Python and the various libraries installed before starting - or at least know how to get them, completely forgetting of course that I had no idea how to do any of this either!

So firstly go to the Python site, download and install python. I would recommend getting Python3 if you have the choice (although the difference won't make much difference for these tutorials). Just be aware if you're planning on using lots of old libraries or Pygame that you will likely want Python2 instead. Run the installer and, if you see the option, choose to update the system Path variable. (If not don't worry we will solve that later).

Next we're going to get python and pip running from the command line. Why? It will make it far easier to install any other external libraries you might want to use in the future - such as matplotlib, numpy, pyglet, ipython or sympy (all of which I'll use a lot in other tutorials!).  So open a command line (or terminal if you're on a Unix system), type 'python', hit enter and see what happens! If you're on windows, to launch the command prompt, click the windows icon on the bottom left corner of the screen, then type 'cmd' in the search box and hit enter. If you're on OSX go to launchpad in the doc -> Other/Utilities -> Terminal. If you're on linux i'll assume you can either do it already or can google it :p

If the command prompt has changed from a dollar symbol to these: >>> then congratulations! Skip the next bit and go straight to the Installing Pyglet section below (type exit() and hit enter to get rid of the >>> symbols and go back to the normal $ prompt). If instead you got 'python is not a recognised command' then follow the instructions below. (it just means that whilst you have python installed your computer doesn't know where to run it from - we can fix that!)

Windows

Open an new windows explorer window, navigate to your desktop, right click on 'My Computer', choose 'properties', then Advanced System Settings on the left hand side. The click the Environmental Variables button, and from the bottom list scroll down until you find a line called 'Path'. Select it and click 'edit'. Then add "; C:\Python34\Scripts; C:\Python34" without the quotation marks and including the starting semicolon, to the end of the text that's already. Hit save, re-start your command prompt and you're good to go! Alternatively, you can just replace 'pip' in the commands below with:
C:\Python34\Scripts\pip

UPDATE: If you're using python2.7 the the commands above should use Python27 instead of Python34

Other OSs

Google how to change your Path variable. Then add /usr/local/bin/pip and /usr/local/bin to it. Alternatively, you can just replace 'pip' in the commands below with: /usr/local/bin/pip

Installing Pyglet

So now that we have pip installed we can now install a bunch of useful libraries by opening a command prompt and typing: pip install <package_name> i.e. pip install numpy

We're going to install pyglet for Python 3 by using the command:
pip install --upgrade http://pyglet.googlecode.com/archive/tip.zip

The reason the command is so long is that we're using a 'development' bit of code (i.e. code that hasn't been exhaustively tested yet). If we were in Python2.7 then we could just type pip install pyglet  but sadly that won't work here! Now wait a little while while the library downloads and installs for you and then you will be all good to go. 

Final note while the package installs

Think about what you've just done. If you successfully updated your path variable you have just taught your computer how to find and run a particular program from the command prompt. That will work the same for any executable program from your computer - you can run them all from the command line by either changing your Path variable or using the full program location. This turns out to be super useful if/when you end up using the command line a lot. And I say that knowing this is most people's reaction to using the command line:

References


Wednesday, 28 May 2014

Pyglet - moving, rotating and translating images in 2D

Continuing on from the previous tutorial, now we can dig into moving things around in 2D. Why bother learning this when I've covered the same thing in Tkinter tut and when I'm going to cover it again in a pygame tut? Because in pyglet, the same commands for movement and rotation in 2D ALSO APPLY TO 3D, so once we make the jump to 3D we will automatically have access to all the colours, motion and movement actions we're already learned rather than having to grapple with all of them together at once. *

Translation

The translation command takes an object you've created and then moves it by a certain amount. In the previous tut, we controlled the shape and size of our triangle by where we placed it, but imagine if we wanted to move it after we had created it. Then we would have to calculate the location of all the vertices of the object every time it moved. So instead, we create an image close to our origin and use that to control its size and shape and then we move it to where we want in a separate command. This comes in really handy in 3D when figuring out where things are in 3D space and describing that in detail does my head in - so i cheat and use this command instead:

glTranslatef()

The three numbers we pass the method tell OpenGL how far and in what direction we want to move our shape in the order x,y,z. Some helpful tips to remember:
  • x = left or right motion. Negative numbers move left, positive moves right.
  • y = up or down motion. Negative numbers move down, positive moves up.
  • z = motion into or out of the screen (3D only), Negative numbers move into the screen, positive moves out of it.
And yes, because we're in 2D all our transformations currently will leave z =0.0. Try changing it if you like and see what happens, but it's likely your image will disappear because we haven't quite set our system up to display 3D well yet.

Rotation

Rotation is similar to translation except it requires some serious math. So once again, rather than bothering with the details we use an OpenGL command to do all the heavy thinking for us. This time though we have 4 input options:

glRotatef(10.0,0.0,0.0,1.0)

They're:
  1. the number of degrees to move
  2. 1.0 if rotating around the x axis, 0.0 if not
  3. 1.0 if rotating around the y axis, 0.0 if not
  4. 1.0 if rotating around the z axis, 0.0 if not
So a full program making use of both translation and rotation would looks something like:


Note: We've also included 2 other commands 

    glMatrixMode(GL_MODELVIEW)
    glLoadIdentity()

These aren't explicitly necessary - take them out and everything will still work fine. However, once we move into more complicated motion these will become really important, because we're going to have to change between viewpoints and these are the command that 'bring us home' (i.e. back to where we started). For those that are interested, OpenGL works by using a matrix to transform/move/rotate our objects depending of our starting position. If we move to a new location then the same matrix will have a different effect - so we 'come home' by loading the identity matrix in case we've accidentally changed location without realising it. Yes this is why linear algebra is important for computer graphics. No it's not the only reason - but it's one of the prettiest. :) 

Challenge!
Does the order of the translation and the rotation commands matter? Is that weird? Can you figure out why or why not? What happens to your image if you rotate it around the x or y axis? Why?

Movement

Time for another method that I mentioned earlier - update().

def update(dt):
    global rz
    rz += dt * 10
    rz %= 360

ry=0
pyglet.clock.schedule(update)

The update() command every time pyglet's internal clock 'ticks', so if we change the position just a little with every 'tick' as long as it happens fast enough this will look like movement. Here's we're probably better to explain by example:

The details are - we create a global variable called rz (the rotation around the z direction)

Challenge!
What happens if you set rz to be constant??? Why? What would you need to add to get translation working?

*I'm certain this can also be achieved in pygame with or without external libraries (see examples here) and if I ever learn how with that I'll post a tutorial on it.

Intro to pyglet and OpenGL

After taking a look at this awesome Minecraft demo written in Python, I decided to do a little digging and playing with the library it uses to use the OpenGL graphics renderer with the ultimate goal of making a 3D game from scratch as an outreach project. Predictably 'having a play' took quite a while to get working because I'd never seen much OpenGL before so I thought I would put together a step by step guide to getting started for those NOT familiar with OpenGL but with some experience of Python.

Here's a minimum working example of a pyglet window using the openGL commands to draw something simple:

Looking at this step by step, we're importing our library as per usual, creating a window, defining an method called 'on_draw' and then initialising the entire program using the line: pyglet.app.run()

You may notice that the one method we have defined isn't actually called anywhere - that's because it's called automatically within the pyglet.app.run() method. There are other functions that are automatically called as well; update() and on_resize() which we will come back to - but they're there to make things easier for the programmer. They trigger at particular time intervals and when the window is resized (or created) respectively.

Looking at the OpenGL commands, we first declare what shape of object we're going to draw with glBegin(GL_TRIANGLES) (we could also pick GL_POINTS, GL_LINES, GL_QUADS and more)
then we specify the location of the vertices, and then we declare the glEnd() so our program knows we're finished drawing and can start rendering for us.

So what's a vertex? It's a point or a corner. If you're drawing points then the 'vertex' is just the location of the point. If you're drawing more complicated shapes then the vertices are the locations of the corners (so a triangle will have 3, a quad [a shape with 4 sides] will have 4 and a polygon can have as many as you want). Pyglet then 'fills' in the rest of the shape for you by filling in the locations between the corners. So what's with the '2i' bit? That's how many dimensions your drawing in. In this tutorial (for simplicity's sake) we will stick to making and moving things in 2D but the next tut will show you how to generalise into 3D.

Challenge!
Using only what I've used above, a couple of other commands below, the power of google (and educated guesses!) try making something impressive out of ONLY the simple shapes described above.
One point to know is that the location (0,0) corresponds to the lower left-hand corner of your window, hence if your window is 400 x 400 pixels, the then upper-right corner is going to be at location (400,400), the upper-left at (0,400) and the lower-right at (400, 0).



Colours
Everything is more impressive with colours, and our image is sorely lacking them. Adding colours in is simple - just add in the commands, and play with the numbers to change the colours:

That said, if you have overlapping triangles then colouring can be a real pain - so if you want to just display outlines of your triangles (to start with anyway) just put in this command and then comment it out when you don't need it.



Challenge!
What happens if you give different corners of the same triangle different colours?

Shortcuts
Ok, defining every point by trial and error quickly gets really tiresome. Conveniently there's a few shortcut commands to speed up the process thanks to pyglet:

This command cuts out the need for the glBegin and glEnd and the individual vertex commands and just gives pyglet a list of locations to connect into a triangle. Here's a working example of its use:
Methods
To do really cool stuff with pyglet (and programming in general!) we're going to have to wrap our code up into nice little chunks called 'methods' which we can call again and again rather than copy-pasting our code all the time. The on_draw bit from the code above is a method that is run every time the program loops. We're going to define another one for making triangles:
Try including this in your code (Hint: put the method BEFORE the on_draw function and call the method INSIDE the on_draw function instead of the current code we have in there).

Extension Challenge!
Given your experience with these shapes now - what's the most complicated design/shape/image you can represent using only triangles? Here's a clue - the below image is called a Voronoi tessellation (google it - there's some amazing art out there based on this technique).

Next tutorial we will use the OpenGL rotation and motion tools in 2D

References:
http://www.pyglet.org/doc/api/pyglet.graphics-module.html
https://github.com/fogleman/Minecraft
http://stackoverflow.com/questions/7681899/moving-an-image-around-in-3d-space
http://stackoverflow.com/questions/4269079/mixing-2d-and-3d-in-opengl-using-pyglet






Sorting emails and spreadsheets with Google Apps Script

I get far too may emails that I can deal with on a daily basis so I use Gmail filters to help me keep track of what is where. However sometimes you want to extract information from these emails without having to read every single one - for instance if you wanted to keep a database of all the emails you received from a mailing list and their content. Google spreadsheets provides several nice tools to deal with this - most notably their Google Apps Script through which you can access much of the Google toolkit's functionality (i.e. sending emails, generating charts etc). Their scripting language is based on Javascript - so even if you're not familiar with it it's not too hard to pick up with some specific examples.

So diving right in - open a new google spreadsheet and from the file menu at the top select Tools -> Script Editor which will open a new window for you to enter your scripts in. The first thing to note is that this script is currently locked to the spreadsheet you opened it from so if you run the commands:


the script will load the spreadsheet you started from and then set the variable 'lastRow' to the row number of the last row with information in it.

If you want to know what values that variables has you can print it to the console log using the command:
(If you're coming to this from Python this is akin to the print command).
Finally, the commands you want to read from or write to a spreadsheet cell are:

To check everything is working, enter some numbers into the first column of your spreadsheet, then copy and paste the following script into your script editor and then select 'Run' from the file menu. You will have to give the app permission to run - and to see the log output choose Logs from the View menu (The execution transcript from the same menu can also be useful for debugging).

All our script is doing is reading through the spreadsheet, printing out the value it finds and then replacing it with the square of whatever it finds. Simple!....unless you accidentally put text in in which case the script will throw an error. I'll leave you to sort out error handling on your own with the Google Apps Script documentation.

So how do we use this to keep a log of information from emails? Like this:

Just remember - for this to work the emails must have the label 'data_emails' (unless you change the variable in the code) and must be marked as Unread.
Now you have that data in spreadsheet format - you can process/sort it with other scripts, send emails that contain it, or automatically graph it each day. I'll cover some of these in future tutorials

Tuesday, 27 May 2014

Scraping websites with Google spreadsheets

Frequently when working with data collection or visualizations it's useful to get data from an external website that doesn't have an API. There are plenty of tools to help you with this but one of the easiest is certainly google spreadsheets, or you could of course write your own web scraping bot with Python and the Beautiful Soup library (which I will outline in a future tutorial). One thing to note - web scraping is not always legal so make sure to check the website's data use policy before you go scraping it and sharing it with everyone!

To get started go grab a google account and/or login to google documents and create a new google spreadsheet. then go find the URL of the data you want to scrape. To keep things simple I'm going to demonstrate with the highest-grossing films data from Wikipedia. Here's the URL:
 
http://en.wikipedia.org/wiki/List_of_highest-grossing_films#Highest-grossing_films

If you're not familiar with URLs, the # character directs your requests to a specific subset of the page - in this case to the 'highest grossing films' section. If you go to the site you will see a table which is the data we're interested in:



In the top left hand cell of your spreadsheet, use the following command:
 =ImportHtml("http://en.wikipedia.org/wiki/List_of_highest-grossing_films#Highest-grossing_films","Table", 1)

Hit enter and the spreadsheet will slowly populate itself with all the data from the table we're looking at and you can graph it or save it as a .csv or text file for later analysis with another program or script.

If it didn't work there's a few things to check:
  1. Make sure you have enough of the right kind of quotation marks. There should be double quotes around both the URL and the word Table in the above command
  2. Check the URL is correct.
So what precisely is happening? The =ImportHtml() command directs google spreadsheets to the particular page and the next command "Table" tells it too look for an element in the format of a table. You can also use alternative Html elements here like lists ("List"). Finally the value 1 tells it which element to look at. Start with 0 and keep increasing the number by 1 until you find the data you're looking for. 

Have a go changing the number in the above command and see if you can scrape the table from further down the page with films adjusted for inflation.


There's a functional example of the spreadsheet here if you're interested.

It's also worth noting that Google spreadsheet come with several other import tools for importing different types of data including .xml with:
  • ImportHtml()
  • ImportXML()
  • ImportData()
  • ImportFeed()
Finally you can also use the other google spreadsheet commands to sort through the data you get. One of the most useful is the =SPLIT() command that separates the incoming data by a certain character (so the command =SPLIT(A1, " ") would split whatever text is in the cell A1 wherever there was a white space.)  Another useful command is =REGEXEXTRACT(), which searches a sting for a particular pattern of letters and numbers known as a regular expression, and then returns it. Regex commands are a little complicated to get used to but really useful - here's one that searches for the pattern of a year(i.e. 2014) in a string:  
=REGEXEXTRACT(A1,"[0-9]{4}"
)