Python 2.6 & MySQL

4 02 2009

For any of you who both run Windows and use MySQL as your database backend, you may have found MySQLdb already.  You may also notice that there is no release for Python 2.6.x on the page.  That struck me as odd, since Python has moved into the 2.6 days near the end of 2008, and then shortly thereafter, they announced and release Python 3.  Why hasn’t MySQLdb caught up?

So I’m stuck with Python 2.5.4?  you say.  Not quite– If you run Linux, just compile the MySQLdb source on your 2.6 version of Python, or if you run windows (and heaven knows it’s hell trying to compile anything from source on a windows machine) then you should just download this obscure 2.6 release of MySQLdb.  The only reason I know about it is because I’m on the mod_python mailing list, and someone had a question about the missing 2.6 version.

So that you’re properly warned, Python 2.6 deprecated the Sets module from ImmutableSets, which gives a warning when MySQLdb gets fired up.  MySQLdb works just fine still, but it’ll likely need some work to become compatible with Python 3.



6 01 2009

Just wanted to put in a plug for Netbeans, the opensource IDE for multiple languages.  And I want to make sure that it’s understood that Netbeans kicks the trash of its alternative Eclipse.  If it’s been a while since you’ve used Netbeans, I suggest you return to it and check it out, for it no longer blows as it once did.

For anybody who walks the 3d-animation walk, Eclipse is the Blender of IDEs; powerful, perhaps, but you have to step back and ask yourself:  Will I ever *actually* learn the interface fluently? It’s an honest question– is it possible that the interface is (gasp) not intuitive??  I submit the answer is a resounding YES.  Sure, there are gurus out there who can whip of crazy nonsenese with Blender, but will you ever be one of them when you have to work with an interface like that?

For those who have issues with certain programming languages out there– be it Java (raises my hand), C/++, PHP, Ruby, Perl, Python, whatever– have you ever been annoyed at the absurd steps involved to get functionality that *should* be basic?  For instance, PHP’s ‘implode’ and ‘explode’ functions:  useful stuff!  Or it’s ‘var_dump’ command!! Holy crap that’s nice.  It’s annoying in Java or C when you want to accomplish the same thing.  Or when in Java you use Hashes (of any kind),  Trees, or other strutures backed by a common class of ‘Collection’ or ‘Set’ or ‘Map’, and then you go to some other language and you ask yourself, Why doesn’t this language have useful tools built into it?

And so the punchline:

Have you ever had a proficient user of any of the above say to you [in/directly] that “oh, it’s easy once you’ve gotten used to it.” ??

… May I interject that “getting used to it” is precisely the problem?!


Netbeans is a great alternative to Eclipse.  While Eclipse may be powerful, I would please like it to be able to detect my JRE for me.  I would like it to not be obscure in its menuing options.  I would like to easily find the option for displaying line numbers.  I would like for the ‘Preferences’ menu  NOOOOT to be found under the ‘Window’ menu (WTF? What. The.  Freakin. F.).  I would like for it not to skin itself all magically, wasting my processing power.  I would like for Eclipse to not have a retarded intro screen.  I would like for its updater/module downloader to give me the option of including dependencies of selected packages, because that list if freaking huge and I can’t find the dependencies it whines about.  I would like for there to be an ‘all-in-one’ download, so that I’m not forces to go to their site and download the Java version and then be all confused when I want to have the C/other plugins.  I would like for the download on the site to have the most up-to-date version of the software, so that when I go to get a plugin, it doesn’t tell me that I need the most up-to-date version of Eclipse, promptly after I have just downloaded what I thought to be such.  I would like for the random package errors in Java projects to go away, as when I import a project that might have a package error, and then it effects all other projects in my workspace.  I would like for run-settings to be project specific, instead of one project hijacking another project’s settings.


I could go on, but I hope that I’ve illustrated the point.  Netbeans is easier.  Far more effectively organized, and far more to-the-point.

Blender may be powerful, but wouldn’t it help everyone if they’d just make it easier to use?  Likewise, wouldn’t Eclipse benefit from an easier interface that acted a little more smartly?  I declare that Netbeans has done just that.  (And it comes loaded with Glassfish and Tomcat if you want them!)

A toast, to cutting the crap!

Java: Count instances of a substring

6 01 2009

Despite my distaste for Java, I work in it every day at e-kiwi (author of software Screen-Scraper).

As usual, I was surprised to find that Java has no easy way to count the number of occurances of a substring within a given String variable.  I Google’d it and was surprised at the first result’s poor coding.  It had a ‘while (true)‘ loop with a goofy break statement tied to a misleadingly-named variable.

Try this:

public int countIndexOf(String text, String search)
    int count = 0;
    for (int fromIndex = 0; fromIndex > -1; count++)
        fromIndex = text.indexOf(search, fromIndex + ((count > 0) ? 1 : 0));
    return count - 1;

Pardon the ternary operator in there, but Java doesn’t like making boolean-casting-to-int easy.

If there are any errors you encounter, just let me know and I’ll fix it.  My use for such a function doesn’t require the ‘border’ cases where substrings are at the beginning or end of the main string.  If you know that you don’t need to match a substring which will begin at the first character of the main string, you can change the ternary part from:
fromIndex + ((count > 0) ? : 0)
to just
fromIndex + 1

Sorry to the dude that I flamed on his blog, but poorly written code is easy to find these days.  Idioms are the way to go.  Java’s slow enough as an interpretted language that it doesn’t need extraneous if-else statements in ‘while (true)’ loops and incessant variable assignment to immutable values.

You might argue that it doesn’t really matter if you lose a few cycles on today’s machines, but programs should be as efficient as possible when it’s worth the time.  You may as well take a few minutes (or even an hour) and figure out the best way to do something.  Forethought with save you an amazing amount of time in the future when you develope real programs and not just school assignments.

Python, MySQL, and a dictionary

19 12 2008

Here is a line in python that should be recorded for posterity…

Some background.  I have a multilingual database where several tables exist.  Each table is named after a particular “vocabulary” for my website.  For instance, there’s a “forums” table for all the terms that are used for pages related to the forums.  The purpose of having this table is so that I have all the needed terms very isolated into a nice hierarchy.  The SELECT * sql would look like this.. (pardon the fake spanish translations)

mysql> SELECT * FROM forum;
| ID                  | enUS                       | es                           |
| Audio Review        | Audio Review               | [Audio Review]               |
| by                  | by                         | [by]                         |
| complete            | completed                  | [completed]                  |
| current             | current                    | [current projects]           |
| dropped             | abandoned                  | [dropped]                    |
| loading             | Loading...                 | [Loading...]                 |
| no                  | no                         | no                           |

As you can see, there is a column “ID”, which is what I use site-wide to access a term, and I select only one other column, according to the language code the user has his/her preference set for.  Thus, a general select for such a thing is …

mysql> SELECT ID,enUS FROM forum;
| ID                  | enUS                       |
| Audio Review        | Audio Review               |
| by                  | by                         |

… etc, etc.  Then, I just used PHP to fetch these results into an associative array.  I would then have to loop through each record, saving whatever is in the $row[‘ID’] spot as a key to this array, and whatever is in the $row[currentLanguage] spot as the value to said key.

So anyway..  I was trying to achive this effect in python; a Dictionary-type object where I can simply index into it with the term that I want, in order to get the language-translated value represented by the term.  I knew there had to be a way to do a famous python one-liner, and since those one-liners typically clock much faster than manual loops..

resultSet = (map(None, row.values()) for row in rows)

Seems simple, but it’s a hassle to work it out when your data is a tuple of dictionaries 😛

Java: Array Slices

14 11 2008

First off, let me start by saying that I don’t really like Java.  We’re working from that assumption, understand? 🙂 

That being said, I have to work in it every day at ekiwi while we develope web “scrapes” for clients.  Our screen-scraper software can utilize Interpreted Java (via Beanshell), Python, JavaScript, VBscript, Jscript, and probably a few more that I’m forgetting to mention.  However, to maintain a standard, we write our scraping scripts in Java.

Given the fact that I think Java is rediculously too verbose, I was happy to find a way to at least partially get the effect of PHP or Python array slices.  Here it goes. Since the JDK 1.2, you can use the following code.

The senario is that we have one array, and we want to cut it into 2 halves. Theoretically, you could just copy the first half, or the second half, or any segment in the middle, so long as you have indexes to the array.

// Interpreted Java
String[] myArray = new String[ 11 ];

// Get the sizes for the arrays.
int firstHalfSize = myArray.length / 2;
int secondHalfSize = myArray.length - firstHalfSize;

String[] firstHalf = new String[ firstHalfSize ];
String[] secondHalf = new String[ secondHalfSize ];

// Arguments are: sourceArray, sourceStartIndex, destinationArray, destinationStartIndex, numElementsToCopy
System.arraycopy( myArray, 0, firstHalf, 0, firstHalfSize );
System.arraycopy( myArray, firstHalfSize, secondHalf, 0, secondHalfSize );

The “System.arraycopy” method is very efficient; it’ll beat any loop you can write in your own code.  (This is precisely why forward-thinking sripting languages like Python build this functionality into arrays from the start with a syntax like “myArray[ startIndex : length ]” (the colon is literal).)

So what does this mean?  Use it.  Bring happiness to those people who wind up using Java by speeding up an overly-verbose interpreted language.

Shortest angle from ‘current’ to ‘desired’ angle

6 08 2008

I figured someone might Google this one one day:

While doing some 3d programming in Python, for FiniteDreams, I was trying to rotate an object from whichever direction that it is currently facing, toward a new desired direction.  I’m only really concerned with “heading“, or “yaw“, which does the side-to-side sweeping in a Z-up coordinate system.

I have a velocity vector built each frame, influenced by whichever combination of directional movement keys are being held.  In order to get my character (or any object with such velocity) to gradually point in the right direction, I had to convert the directional velocity vector into some “heading” value:

>>> from math import atan2
>>> newHeadingAngle = atan2(newVelocityVector.getY(),newVelocityVector.getX()) * (180/pi)

As you can see, atan2 must be imported from the math library.

Next, I have to find the shortest angle between where my current angle is, and where I want to go.  There’s  no sense making my character turn 300 degrees clockwise if he only has to move 60 degrees counter-clockwise:

>>> shortestAngle = newHeadingAngle - currentAngle
>>> if shortestAngle > 180:
...    shortestAngle -= 360
>>> if shortestAngle < -180:
...    shortestAngle += 360

And voila.  Now you’ve got the angle in the range [-180,180], acording to the direction that the object needs to rotate to get to it’s desired heading.

But, we don’t just want to instantly make the character face that direction.  Instead, s/he needs to gradually turn there.  The advantage that we now have is that we have already calculated an angle that needs to be traveled.  All that remains to do is move the character toward it each frame:

>>> movementThisFrame = self.shortestAngle * characterAttribute_speed * elapsedTime
>>> self.shortestAngle -= movementThisFrame
>>> characterNodePath.setH(characterNodePath.getH() + movementThisFrame)

Note that I’ve used the “self” syntax this time around, since I’m getting more specific to the panda3d code.  The “self” variables are persistent in a class object; the context of this code is a panda3d “task“, which is a method that gets executed each frame.  Thus, I need to retain certain of my calculations.  In this case, I need to remember that “shortestAngle” variable, so that each frame can move closer and closer towards it.

The first line takes the character’s speed into account, so that I can have a big fat character turn more slowly than a thin one.  The “elapsedTime” variable is the total time elapsed since the end of the last frame (which I’ve calculated myself).  This regulates the speed at which things happen.  If I do not include this variable, the game would run at terrifyingly unpredictable speeds on different computers, depending on the hardware.

Line 2 subtracts the distance that we want to move this frame, from  total angle that we would like to move towards.

Line 3 actually performs the rotation.  This line does not make a smooth movement out of my new angle, which is why I’m calculating a “movementThisFrame” variable, which is quite small compared to the total angle.  Since it is small, line 3 doesn’t have the visual effect of being choppy or chunky.

Also important is to remember to reduce down the degree of rotation that the character is currently in.  Even though the target angle is reduced, adding an angle of 178 degrees to an existing angle of 164 will add up to 342 degrees of rotation.  The model will still be pointing the right direction, but to stop from winding dozens of times into higher and higher angle numbers, this should be reduced via the same angle-checking method used above, for the “shortestAngle” snippet, two code blocks above.  The only difference would be to use “characterNodePath.setH(characterNodePath.getH() +/- 360)”  If done every frame, the character will always appear to move and point in the right direction.

As one last note, I’ve made a “if” test on the remaining “self.shortestAngle” variable, so that if the remaining angle distance to be covered is less than 0.05 then I just zero it out.

Python basics

5 08 2008

Python‘s great, really. It takes a little bit of a mode-change before you get used to it, but I’m really quite taken with it. I’ve got experience with C/++, Java, Perl, PHP, Javascript, and a little in Ruby (and I can hammer things out in a number of others, but I wouldn’t claim to “know” the language), and I’m simply impressed with Python.One major plus: Python was built to be a command-line tool.  Thus, you can “run” Python and it will be in command-line form, where you can enter one line at a time.  Sounds rather daft until you’re sitting on a hard chair at 3am trying to guess at how a method behaves.  Python makes this sort of guesswork trivial, as you can just open the command prompt (or terminal, if you’re on some Unix-based system) and run “python”.  You have the ability to simply test a command several times over, to observe its behavior.This INVALUABLE to a person like myself.  There are times when (let’s face it), the online documentation is a bunch of crap, or (in my case), the documentation on the panda3d library’s methods is incomplete.  You have the power to just *do* the command and see for yourself, rather than trying to debug your current program for strange behavior, or trying to built a seperate program for the sole purpose of testing a function or method.  I’ll be honest, I usually have a Python process running in my command prompt while I program in my IDE off on the other monitor.

Some quick observations about Python: 

  • Global scoping can be weird.  Try to be as encapsulated as possible.  Python’s scoping is more like “static” (if you’re familiar with the C++ idea of that term).  There’s no cross-file variable/function/method sharing.
  • Functions” are created with the “def” keyword.
  • Any functions (“def”s) that are within a Class always take “self” as their first arguement, by convention.  When you call these functions/methods, they do NOT require that you pass an arguement to fill the “self” variable in your declaration’s arguement list.  This is resolved automatically.  It is like the “this” pointer of C++.  It’s a reference to the class object that the “def” belongs to.
  • Any variables that should be persistently available in a class should be accessed by the “self.myVar” syntax.
  • A method defined by “def __init__(self):” will act as the class’s constructor.
  • Triple-quoted strings (using three single quote or double quote characters in a row) can be used as something called a “docstring“, and should be placed directly under each class and def statement’s first line.  Triple-quoted strings are really just immune to linebreaks.  That’s all that’s special about  them.  They just happen to be used for this “docstring” very commonly.
  • There are no braces in Python code. Indentation is the way that Python knows what block you’re in.  All statements that should be within an “if” block should be indented the same amount–no more, no less.
  • General lines do not require a semicolon at the end. This is because Python was made for line-by-line console programming.  Making every line end with a “;” is a silly notion.  Just push “enter” on the prompt to finish the line.  However, including a “;” at the end allows you to execute multiple statements on a single line–functionality only really useful from the command prompt.
  • Blocked statements, like “if”, “else”, “elif”, “while”, “for”, “def”, “class”, etc, require a colon at the end. This seems utterly backwards to the normal programming standards out there, but again, on the console, it makes sense; this is how Python knows that you want to enter more information to be within the block that you’ve just tried to define.
  • Everything in Python is by reference. Sounds annoying at first, but it makes the programs a bit more lightweight on the memory footprint.  Variable assignment, if dealing with Objects, is by reference:
    >>> a = myClassObject()
    >>> b = a
    >>> b.setX(3)
    >>> a.getX()
  • Comments are made with a pound, or hash, sign “#”.  There are no multi-line comments.  You could, however, be a jacker and abuse the triple-quote string to get multi-line “comments” in your code.  It’s nicer just to put a “#” at the front of each line.  C’mon… how lazy ARE you?

And those are some of the basics.