I usually use prepackaged binaries to install Python on Windows machines. Lately I have been trying to get back into using Intel’s open source vision library, OpenCV. To do that it appears I need to compile extension modules to be able to use Python, which in turn requires that I have the source and be able to build from it.

The build process has been relatively smooth, the MSVS 2005 solution file is located in the PCBuild8 directory. The biggest hiccup has been to get the sqlite3 extension module compiled, the instructions say I need to download the source distribution for sqlite3 from the svn.python.org SVN repository, but it has not been available today so far. Building sqlite3 from sources available at the sqlite site is a hassle, as it appears to require Mingw. I can of course do this, but I don’t have the time I used to.

A shortcut is to download the binary/DLL files from the sqlite website. This zip file contains both sqlite3.dll and the DEF file. However, the python solution requires a sqlite3.lib file, which is usually generated when one compiles and links the source files.

A little bit of googling led me to the def2lib.exe tool which converts DEF files to LIB files. The tool is located here.

It would not come as much of a shock to most if I said engineers in general get comfortable with particular tools of the trade (microcontrollers, languages, chips, etc.) and resist change as much as possible. This is not without some good reason, as all the time spent getting familiar with the published and unpublished “features” of a particular system are sunk costs that precludes one from examining more than a few alternatives.

 I am like this, and my comfort zone currently in terms of embedded systems is the Cypress PSOC. I like its design tools (PSOC Designer, not the PSOC Express), like the analog/digital blocks that can be reconfigured, and even the much maligned M8C core. (Its assembly instruction set is very similar to my all-time favorite, the 6502)

 However, even Cypress has realized that the M8C 8 bit core is a little underpowered. Apparently they have been designing the next generation (PSOC3) chip that utilizes an ARM Cortex M3 core, (and another version with an 8051 core). I’ve heard good things about the ARM series, but since they were 16/32 bit systems with a huge development suite, I resisted. Not to mention the cost of a beginning development board/system was much higher than for the PSOC.

With Cypress’s move to the ARM architecture, however, I have decided to sit down and learn more about ARMs. PSOC3’s haven’t come out yet, but I am thinking of getting an ARM SAM7 board from my favorite PCB fab company, Olimex. The SAM7-EX256 board looks pretty feature-full, and it even includes a color LCD to boot (this year’s mobile robotics course has taught me the necessity of having an LCD for debugging purposes).

The price is a bit steep ($199) but I think well worth it. Fortunately, there is an open source development suite, the WinARM that includes gcc, Eclipse, and a gdb/insight based debugger. All in all a “professional” development environment. I don’t have enough time to learn yet another assembly language, so being able to use a (free) C compiler is practically a requirement.

More on my ARM adventures once I get the board.

I teach another course, an embedded programming and robotics course. For the embedded programming part I use the Cypress Semiconductor PSOC (Programmable System On a Chip), which is a really versatile microcontroller/SOC.

 I have been trying to interface an ultrasonic transducer to the PSOC using the Timer user module, and came across an error in the datasheet. I posted the following on www.psocdeveloper.com, a website that is devoted to the PSOC, but didn’t receive an answer. Luckily I was able to figure it out myself!

I am examining the output of the Device configurator
for a Timer16 module, for both InvertCapture set to
Normal and Invert.Now the Datasheet says that the
"Data Invert" Bit exists in Bit 7 of the Bank 1 MSB
Function Register.Looking at the generated
psocconfigtbl.asm file, it appears that the bit that
is being set/cleared by the Designer is the LSB
Function RegisterInvertCapture Normal:

; Instance name T, Block Name TIMER16_LSB(DCB22)
db 48h, 00h ;T_FUNC_LSB_REG(DCB22FN)
; Instance name T, Block Name TIMER16_MSB(DCB23)
db 4ch, 20h ;T_FUNC_MSB_REG(DCB23FN)
InvertCapture Inverted:
; Instance name T, Block Name TIMER16_LSB(DCB22)
db 48h, 80h ;T_FUNC_LSB_REG(DCB22FN)
; Instance name T, Block Name TIMER16_MSB(DCB23)
db 4ch, 20h ;T_FUNC_MSB_REG(DCB23FN)

Does anyone know which is correct, the IDE or the datasheet?
I am using 4.3 and have checked the 4.4 release notes but
don't see this mentioned at all.

 I posted the solution as well:

The data sheet is indeed incorrect and the IDE does
generate the proper code (LSB). what was preventing
me from quickly deducing this was a bug in my own code,
which failed to set the bank to 1 before changing the
register.It now works and I can dynamically flip between
normal and inverted capture.

The code I wrote (and fixed) is:
Ping_Capture_Normal:
 M8C_SetBank1
 mov reg[PingT_FUNC_LSB_REG],Ping_CAPTURE_NORMAL
 M8C_SetBank0
 ret
Ping_Capture_Inverted:
 M8C_SetBank1
 mov reg[PingT_FUNC_LSB_REG],Ping_CAPTURE_INVERTED
 M8C_SetBank0
 ret

I just caught a very important security issue that I had missed! The SQLPAS PlonePAS module requires a database connection to the user/password tables. Since you can’t add a Z MySQL DB Connection (or any dbc for that matter) in the acl_users folder, I left it at the top level root directory. This unfortunately has the effect of making that database connection usable to students in the Members directory, by acquisition! They could conceivably then do a SHOW TABLES, etc to find out student info, including grades!

The solution was to create a Z MySQL DB Connection with the exact same id and create it in the Members folder. That way, if a student creates a Z SQL Method the only connection that he/she sees would be the one to the test database, not the student info db!

Well I finally have it working the way I want.

Rundown of flow:

• Student creates PSFolder
• Student submits PSFolder
• PSFolder gets moved to separate folder that only TFs can access
• TF logs onto account and goes to tf_resources and selects View Submitted Homework
• TF grades each submitted PSFolder
• When done, TF returns to student PSFolder
• PSFolder gets moved back to student folder

New features to be added eventually

• Incorporate grading pages to enter grades while in workflow (this is complicated)
• Incorporate archiving of submitted folder in case of disputes later on
• Email on homework submission/return

Some hard lessons learned

• If you manage_pasteObjects(manage_cutObjects()) requires both Copy or Move and Delete permissions (need to confirm)
• manage_cutObjects() takes a list (or tuple?) of ids, not an id. (sort of obvious)
• portal_catalog gets confused if you do a copy -> paste. do a cut -> paste. Confused means the review_state variable is “private” (the original state) not “submitted”
• if as a student you can no longer see your home folder, that means some workflow permissions were screwed up, just reset security settings in portal_workflow to clear this up
• put the object move script in the After section, not Before. otherwise portal_catalog gets confused too, as the state hasnt changed to submitted yet.
• Properties, based on a db using SQLPASPlugin, need to be set through Plone using setMemberProperties, not directly into the db.
  • I created an external method script to do this. extSetMemberProperty // extSetStudentTFStatus

Parts of the workflow system

• Portal_catalog: Add index/metadata for TF
• Portal_workflow: Add PSFolder type
  • Add Variable TF
  • For submit transition use expression: TF = python:here.portal_membership.getAuthenticatedMember().getProperty(‘tf’)
  • Set Script (after) to an external method that handles the move extMoveObjectsPlone
  • For return transition set script (after) to extMoveBackSubmittedPlone
  • Both are in the external file extCopyObjects.py
  • Permissions for the submitted state: Owner should have Access contents, Copy or Move, and Delete
  • You need to edit the Adds to actions box and add a label or else the transition won’t appear (e.g. “Submit to TF”)
• PSFolders are Plone objects, not ZMI/zclass objects. Create a regular Plone folder to store the moved psfolder objects in the top level portal root, and manually create Plone folders with tf user ids as ids. Make sure this folder is Private!
• Set permission to private, submitted, graded, etc appropriately.

I’ve added code snippets to the svn server, which can be accessed on trac.reisun.com

I’ve been having some difficulty implementing student homework submission. I’ve gotten a crude hack working, but trying to improve it has been more difficult than I imagined. The following excerpt from http://www.zopelabs.com/cookbook/1029298314 will help, I wanted to copy it here so I can refer to it later. (For a while I had forgotten how to access this page!)


I used your idea but ran into a problem because the default action after a change in status is to view the object (at the old location), which results in an object-not-found error.To fix this, I went into portal_properties/navigation and changed default.content_status_modify.success from "action:view" to "url:../folder_contents" Now when I publish an item, it dumps me into a view of the folder where it came from.It would be nice if it would go to a view of the newly-moved object, but I haven't been able to get necessary redirects working.Fixing Plone's response (was Re: Re: Use one script for many types of objects.) by sh23 - 2004-10-14
I wanted to only be placed in the directory when the object was no
longer there. In /Plone/portal_properties/navigation_properties
instead of modifying default.content_status_modify.success, I created
a new entry:

I have been trying to convert my course website code to work with the new Plone 2.5.2 install. Typically, new features aren’t documented clearly or at all. The biggest hangup is the PlonePAS setup, that has changed the way membership data is accessed. No longer do portal_membership calls work, or at least not completely.  For example, context.portal_membership.listMemberIds() works, but only for the users defined in the default member plugin. I use SQLPASPlugin, and the above code does not list members defined in an SQL database.

The only way I’ve been able to get access to them programmatically is to use context.acl_users.X which I am not sure is the right way. Some test code below:

# Test code for new PAS Plugin architecture---NOT hooked into portal_membership
# print "Current User: ",context.portal_membership.getAuthenticatedMember()
# print "Users: ",context.portal_membership.listMemberIds()
# print "Search Users: ",context.portal_membership.searchForMembers(name='staff1')[0].getRoles()
# print "Search Roles: ",context.portal_membership.searchForMembers(roles='cscishstaff')
# print "acl_users: "
# users=context.acl_users.searchUsers(roles='staff')
# for u in users:
#     print "%s(%s)"%(u['userid'],u['title'])
#     print u
# return printed


I had hoped that I was done with installing ZMySQLDA on Plone, but it turns out there were some additional hiccups. ZMySQLDA was being recognized as a product in the Control Panel, but it disappeared from the drop down menu again. Looking at the event.log revealed that it was trying to extract files from an egg file into my personal user home directory, not the “plone” user that was created and under which the plone process was being run. Not exactly sure why this is happening, but I took the lazy way out and manually copied and pasted the MySQLdb folder and _mysql.so file into the Python directory. An added complication was that the _mysql.so had to be also copied into the ZMySQLDA directory.

 I really need to sit down and figure out all this egg and process configuration out, but am running out of time setting up for my summer courses.

I have been ignoring going into details of web security in my introductory web class, partly because we cover so much (HTML, CSS, Javascript, SQL, python) that we really don’t have time to cover it, and partly because I’ve been too busy to come up with some slides for it.

With the prevalence of XSS (Cross Site Scripting) hacks, it looks like this year I will be spending at least a few slides on security.

Some resources:

• http://www.cgisecurity.com/articles/xss-faq.shtml
• http://www.cert.org/tech_tips/cgi_metacharacters.html

An informative post on slashdot outlined a few points to consider when designing a site to be robust against XSS (I’ve highlighted his points, but the words are mostly his):

by Fireflymantis (670938)on Friday June 15, @04:21AM (#19516455)

•  Are you making sure, without fail that if a user changes view.php?id=32 to view.php?id=33 that they are not getting access to content they shouldn’t be?

• What about cookies? Assuming the malicious user can (and will) build cookies of their choosing and content, are you making sure that this cannot somehow be used to hijack another users account?
• Are you 100% certain, that every time you read get/post/put data that it has been marked as tainted, validated, and only after it has made it through some very harsh sanity checks it is allowed any where /close/ to a DB insert/query?
• It gets even more muddeled in the world of XmlHttprequests when you have to validate against a plethora of other constraints
• Simply checking form data is almost 99% of the time not enough. For a non-trivial web app, even the above is not easy to do unless you pay attention to it every step of development. And even if you do that, you will probabaly miss something.

I’ve just spent the last 2 hours or so installing ZMySQLDA on a Plone 2.5.3 site. It was considerably more painful than I thought, as I never used to have troubles with this product before.

The problem? PIL is required to be installed for Plone, and its ImageFile package conflicts with a similarly named package that is part of Zope (App.ImageFile)

 This is fixed in the DA.py file of ZMySQLDA by explicitly importing the App.ImageFile package, not the PIL version.

 After having the ZMySQLDA product finally appear in the Control Panel, it still wasn’t showing up on the drop down “add” menu. I am not sure what I did to fix this, but for a while only a single ZEO client had the menu option visible. Although I had rebooted/restarted the ZEO client/server multuple times and multiple ways (through the control panel and using the restartcluster.sh script) I think I still had the problem.

In anycase, it now works and I have both ZEO clients showing “Add Z MySQL Database Connection.”

I had planned on getting SQLPASPlugin setup, but it is already 1227, and I need to go to bed.