Isn’t IT supposed to save money?

From this NY Times article  the New York City Department of Education (DOE) is spending $700 million on a payroll system. You know the type of thing I made as a hobby or any number of existing projects. Meanwhile teachers are being laid off. There’s some real administrative problems here, but there’s also some very big IT ones when $700 million is spent on effectively nothing. I would take a project like that for $50k plus hardware expenses, but actually I’d feel bad ripping of the DOE.

IT is supposed to save money remember? Having machines work for you. If this isn’t happening you are doing it wrong. Last year I made a proposal to a certain school in New York to create a free and open source information system. It keeps payroll of student workers. It handles grades and general school data. It even integrates with products like SugarCRM and Engrade (free but not FOSS). The program brings what would have costs over $5000 a year to just $2000, a fee I charge for continued support. But the real savings is in efficiency. Student Worker Relational Database is saving time for teachers and staff. Remember, that’s what IT is supposed to do, computers working for you. Teachers can look up student data in seconds, rather than pour over various databases and Excel sheets. Principles can do complicated analytics in minutes rather than days.

[Student Worker Database] brings together all the information that different people in the school -Administration, Teachers, CWSP, College Counselors, Dean of Student, Social Workers, etc – used to keep in different and not always integrated software. I can now have all this information integrated in one common report saving vary valuable time. The grade analytic feature has also given us the possibility of accessing and processing data much faster and using it to inform teachers, students and parents.
– Maria Andreau, Assistant principle of Cristo Rey New York High School

The open source model allows other schools to potentially contribute as well. Unlike some industries, education isn’t really about “beating” your competition. Yes every school wants to be the best, but that doesn’t mean you want other schools to fail. If another school benefits from work done on an open source project, it actually benefits the early adopter school. That’s why I don’t believe proprietary systems have any place in the education environment. There may be some models that it makes sense, but not schools. Schools are the perfect environment for open source. They aren’t as competitive. They have very common needs. They attract people who are in it to help people, not just the money (which isn’t so glamorous). But where is open source in the industry? It’s near non existent. There are some exceptions, but most schools use Windows, MS Office, proprietary school information systems, proprietary fund raising tools, etc.

If there is a beacon of hope, it’s that these proprietary systems are awful. The opportunity for a hugely successful open source project is here. The information system I replaced with year was a 90’s DOS application ported to Windows. Updates generally include a new picture and marketing about how modern the system is. Even the newer web based systems are expensive and don’t offer integration outside their select partners.

I have more ideas, stay tuned!

Linux in schools

My most popular post is the Linux and Active Directory post. Today I’ll give an update to the project and what advantages and disadvantages Linux has. The project was to set up a trail run of Linux at the school I work at, in the hopes that by the discontinuation of Windows XP, we could move fully to Linux. Currently there are around 30 Netbooks, 4 desktops, and a mostly Linux servers with one Windows application server.


In a short survey I found most people believed Linux to be faster. People also generally thought Linux was saving money. On two Pentium 4 desktops, memory was upgraded to 1GB and Ubuntu was installed. The difference was night and day in terms of performance. These computers would have had to been replaced otherwise. Surprisingly to me, users reported the Ubuntu machine to be much more stable! I’m probably a bit jaded after one too many x server crashes while switching monitors. I suspect I have more problems because I have higher demands, most work users don’t care about PPA’s and proprietary video card drivers. While not directly related, I’ve been installing Ubuntu on a considerable number of student’s laptops as they bring them in broken. I feel exposing them to an alternative to the consumer culture of buying the new version/computer every year is valuable. Most student’s don’t know there are other options than buying or pirating the latest software and replacing their hardware every two years. Ubuntu really shines on simple home applications, where the user just needs hassle free internet access. Being able to clone machines fairly easily is nice too. There only needs to be one image for the entire school. I don’t have the data to prove it yet, but I suspect the long term costs of supporting Linux will be minimal. The machines, once setup right, should be able to just sit and run. Only a few things, like Google Chrome and LibreOffice, really need updated. Chrome has a wonderful repo that it comes with that seems to work on any version.


The time to get the image ready is immense! It’s about 20 times harder to get a Linux image ready than Windows. Printer support sucks. Auto detect never works ever for me. Some commercial laser printers just don’t work. Others require fiddling with the right ppd file. Even when you know what to do, the interface is terrible and involves waiting for it to time out over and over while searching for a driver you know it won’t find. Another problem is there just isn’t anything like folder redirection in Linux. You either have to put everything on nfs or samba. Or use a Dropbox like solution, which there aren’t even any decent open source implementations of. Our teachers need to take home work and have this be seamless. Can’t be done in Linux. Another issue is switching monitors. X will crash. Even one in 20 times is enough to not deploy, since that means lost work. Getting Active Directory support is a pain as previously discussed. The initial joining a domain isn’t hard, it’s all the little bugs and limitations. It also adds an extra minute to startup time.

Any Linux migration will probably involve LibreOffice. LO has problems. Power point import and export is terrible. Also it’s very buggy. It’s nice to have the latest version of LO, since it’s one the most used programs. But each update breaks the default configuration! I have to manually set the default file format each time, which is absurdly difficult.

sudo sed -i ‘s/<prop oor:name=”ooSetupFactoryDefaultFilter”><value>writer8<\/value>/<prop oor:name=”ooSetupFactoryDefaultFilter”><value>MS Word 97<\/value>/g’ /usr/lib/libreoffice/basis3.3/share/registry/writer.xcd

Calc is generally bad. Why does pressing ctrl-down go to row 1048576 instead of the last row with data? Why does EVERYTHING increment. If I type “ID 200” and drag it down, the next becomes “ID 201”, etc.  Auto filtering likes to remove itself when you switch sheets. It has some features though on Microsoft. I honestly don’t see how Excel gets away without having regular expressions. Also MS Office just can’t handle open document files. Students like to have lots of random file formats they somehow make. LibreOffice opens them all, while MS Office can’t even open MS works files! I’ve been a long time OpenOffice user. Recently I’ve been playing with MS Office just to see what others are using. Now I hate all office software. Don’t even get me started on that horrid ribbon.

Is it time to switch?

I have a lot of bad things to say about Linux. But trust me, if I reviewed Windows or OSX, well it would be like watching a Angry Video Game Nerd episode. I have little patience with bad technology. Linux is bad. It sucks. It really really sucks. But so does everything else. Building an operating system and surrounding environment is just hard. No one has it worked out yet. Linux is working out great in student netbooks and administrative staff desktops. I have no plans to deploy it on teacher laptops. No folder redirection and terrible monitor support are the blockers here. I’m really hoping by the end of life for XP, Linux will be better.

One huge factor in allowing me to play with switching users to Linux is web applications. I’m not a sys admin. My real job is a developer and I’ve been replacing costly, unfriendly applications with open source ones, including my own. For users who just need web access, Linux rocks! It’s faster, it’s stable, it’s better. It will reduce support costs and let you get away with not upgrading computers. Lose Outlook, Office, and your legacy Active X web apps, and the switch to Linux is easy. Next post I’ll update my progress on my school information system and future plans for educational administrative software. Stay tuned, I promise it will be more optimistic.


Buy Linux Preinstalled?

I felt like needlessly spending money so I bought a new computer! I got a System 76 Gazelle Professional. It’s a high end laptop that comes with Ubuntu. If you’re looking at buying a computer that comes with Ubuntu you are probably wondering if there is any advantage. Does everything just work at all times perfectly? Yes and no.

The Good

With Linux reinstalled you can expect everything to just work out of box. No downloading proprietary drivers, this machine is made to work, not to appease RMS. It’s really a considerable time saver rather than researching hardware, installing yourself, etc. This is the first computer I’ve ever purchased that the first thing I did wasn’t install a new operating system. It’s close to a stock Ubuntu image, but some good changes have been made. The finger print reader works for one! It’s shocking, I’ve had a finger print reader for years in Linux and it’s always been a situation where it works, but works terribly and so I always disable it. The Gazelle has “fingerprint-gui” installed which works at all prompts. Login, terminal, gksu, even gnome screen saver.

System 76 didn’t put any bloatware unlike any major computer vendor. This makes it a great choice for people who don’t know how to install an operating system. Techie’s probably don’t think about this so much, but really it will make you cry to see a new i5 running slow from all the pre-installed crapware. Of course this works out great for the looters who get people to think they need a new computer every two years.

The Bad
While I generally like this laptop, and it’s way better than buying one with Windows, it doesn’t make Linux perfect. The USB 3.0 ports don’t work right after suspend. NVidia proprietary drivers still suck. There’s really only so much System 76 can do here short of reverse engineering the drivers better than Nouveau, which only gets you about 10% the performance.  The GUI for changing monitors is an abomination.

It has too major issues. First WTF is X server? Obviously if you are reading my blog you probably know. But really a non techie doesn’t give a damn what X Screen 0 is. It’s very unclear how the hell one adds a second monitor using this tool. The gnome tool only works with open source video drivers. Second even for the system admin, it takes 10 clicks to switch to a monitor! That’s just stupid. Also if you remove the second monitor, it doesn’t detect it. If you disable the laptop monitor then unplug the second, you have no way to switch back! What was NVidia thinking!? That said I haven’t had X freeze once so far. My old Intel gpu thinkpad would crash all the time! Ok actually 10% of the time when switching monitors. That’s unacceptable for a supposedly well supported open source video driver. Performance wise the NVidia card works well. I tried out Oil Rush and had no problem running the graphics intense game.

Another point is that you might think a Linux computer would be cheaper since there is no software license fee. This is just not true. Windows licensing for  OEM is very cheap. Even a company like Dell often charges more for Linux computers (because they don’t seem to get included in sales). System 76 is a small company so you can’t expect them to be compete well price wise. You end up with computers that are about the same price as the larger manufacturer’s Windows machines. Also larger companies make money by putting crap on your computer.


Overall I would say it’s worth going for a Linux preinstalled computer. Just don’t expect it to cost less nor to make Linux work flawlessly. It saves you from fighting unsupported hardware. Wading through new hardware with Linux is nightmare filled with land mines such as Nvidia Optimus video cards that don’t work at all in Linux. Also you are supporting a small Linux friendly business rather than throwing money at Microsoft for a product you don’t intend to use. The only exception might be if  you are looking for very specific features. There just aren’t many Linux friendly computer vendors, so if you want a touch screen for example, you may be out of luck.

Linux netbooks fail

A few years ago there were several Linux netbooks on the market from companies like Asus, Dell and HP. I see them from time to time. Every one with the original operating system is broken. I’ve seen no exceptions. If you were a non technical consumer trying out the Linux thing, you probably think it’s a scam that stole $300 from you and are happily back using Windows right now.

What happened?

The PC makers decided to put in 4GB solid state drives to make them as cheap as possible. Next they throw in Ubuntu or some other custom distro on it and offered no support. As soon as a few kernel updates came out the machines filled up with disk space. Thanks to automatic updates or perhaps user who didn’t know what to do, the updates broke leaving unbootable systems. I’ve seen a few. They seem to get to GDM them can’t go further. Logging in brings you the to the log in screen…

Today I see I made the same mistake. I installed Ubuntu one of these broken netbooks. I uninstalled everything I could think of, but still has about 2.6GB of disk space. Now I got the computer back about 2 months later broken again. It tried to do an update, ran out of space, couldn’t boot. Ubuntu keeps old kernels around just in case, it never deletes them. Three updates is all it took to bring it down, it was a ticking time bomb. Now it’s easy to blame the OEM for this, obviously they should have seen it coming. Asus, with their customized Xandros, obviously didn’t care one bit. Some of their systems broke on the first update. Why on Earth should a system update itself to death? It’s sad to think everyone who might have been exposed to Linux with one of these netbooks must have been burned by it. This time I’m disabling updates.

Well here’s an idea #27342

Recipe: Better Than Grandma’s Mac and Cheese

Well a long time ago I said I would include a few recipes just for fun so here it goes.

Mac and Cheese

ready for some mac and cheese

Everyone loves it, it costs little to make, and it’s easy to make in bulk. There’s a million ways to make it, mine’s the best.

  • Noodles (doesn’t even need to be Marconi anything small will do)
  • Onions
  • Cheese (Cheddar, maybe a blend with some others)
  • Milk
  • Flour
  • Beer
  • Butter
  • Breadcrumbs

Boil the noodles, leave them a little underdone. Make a while sauce. There’s plenty of recipes for this basically you just melt butter, then mix in milk and flour and simmer (or add more flour) till it’s the desired consistency. Sauté the onions in olive oil (or anything). You could also throw in other vegies such as peppers, leeks, and tomato. If you have a large enough baking dish you can just mix everything in there, otherwise use a bowl first. Mix the noodles, white sauce, beer (dark or light each gives a very different but great flavor), onions and 3/4 of your cheese. Sprinkle breadcrumbs and remaining cheese on top once it’s in the baking dish. Bake at 350 until the cheese and breadcrumbs starts to turn brown.

You’re notice I didn’t include quantities. You should know what mac and cheese is supposed to look like. The now cheesy white sauce shouldn’t be water nor a solid that doesn’t mix. More beer and milk will make it runnier while cheese and flour will make it more solid. Enjoy, grab a beer, and get back to coding!

Spreadsheet reporting in Django

It’s often desired to quickly export data in a generic method that takes little coding. There are already some solutions for this. I didn’t like any however as they ignore  many to many fields. One could argue a more robust system might be needed when handling more complex reports with gui sql query builders and such. Screw that here’s a hack to get the job 95% done.

I want to check off fields in Django Admin’s model list display. Then make an action to make a spreadsheet report, not some crap csv.

Here is my hack function to take a model and query set and spit out an .xls file

def admin_export_xls(request, app, model, qs=None):
    """ Exports a model to xls.
    qs: optional queryset if only exporting some data"""
    mc = ContentType.objects.get(app_label=app, model=model).model_class()
    wb = xlwt.Workbook()
    ws = wb.add_sheet(unicode(mc._meta.verbose_name_plural))
    #for i, f in enumerate(mc._meta.fields):
    #    ws.write(0,i,
    # Lets get all fields names, even m2m
    for i, field in enumerate(mc._meta.get_all_field_names()):
        ws.write(0,i, field)
    if not qs:
        qs = mc.objects.all()

    for ri, row in enumerate(qs):
        for ci, f in enumerate(mc._meta.get_all_field_names()):
                # terrible way to detect m2m manager
                if unicode(getattr(row, f))[1:51] == 'django.db.models.fields.related.ManyRelatedManager':
                    # If it's a M2M relationship, serialize it and throw it in one cell.
                    value = ""
                    for item in getattr(row, f).all():
                        value += unicode(item) + ", "
                    value = value[:-2]
                    ws.write(ri+1, ci, value)
                    ws.write(ri+1, ci, unicode(getattr(row, f)))
                # happens when the m2m is has an appended _set. This is a hack that works sometimes, it sucks I know
                    f += "_set"
                    value = ""
                    for item in getattr(row, f).all():
                        value += unicode(item) + ", "
                    value = value[:-2]
                    ws.write(ri+1, ci, value)
                    ws.write(ri+1, ci, "")
    fd, fn = tempfile.mkstemp()
    fh = open(fn, 'rb')
    resp =
    response = HttpResponse(resp, mimetype='application/ms-excel')
    response['Content-Disposition'] = 'attachment; filename=%s.xls' % \
    return response

That will serialize manytomany fields to comma separated fields all in one cell. Next in you need an admin aciton.

def export_selected_objects(modeladmin, request, queryset):
    app = queryset[0]._meta.app_label
    model = queryset[0]._meta.module_name
    return admin_export_xls(request, app, model, queryset)
export_selected_objects.short_description = "Export selected items to XLS"

You can see I made mine global. It works on all models. For this to work well you need to make sure your unicode representations of your models are useful. The most common thing I run into are phone numbers. A person can have unlimited phone numbers. The end user will assume a report of people will include this. I make my number’s unicode something like “Cell: 555-555-5555”.

Of course this code isn’t perfect and there are many times a more robust solution will be needed. What if you want a report of companies with contacts at the company and phone numbers of each contact. At that point you need to generate a query that can get such data and that’s going to take some gross gui query builder program or custom reports by you the developer.

Looking for a graphic designer intern

Looking for some help on my student information system SWORD. Fixing up the user interface and making a marketing website.

HTML and CSS by hand
Previous work example

Experience with version control system
Experience with HTML template framework such as Django’s template language
Experience with user interface design for business applications
Experience in open source
Desire to compete with and take down companies who have people pay for their 30 year old software

The pay is…cheap. Like $8/hr for a couple months or the summer. Might be good for a student, maybe I could even put together the paperwork to call it an internship. Could pay some % of revenue made from this. Everything would be done remotely with your own equipment, except servers which I could provide. You would be working directly for me as a contractor for a limited amount of time (unless it suddenly starts making a lot of money).

If interested email david @ this website’s domain or comment. Submit a resume and example work.

CAS in SugarCRM

It’s possible to use Jasig CAS for SugarCRM Authentication.

First install CAS PHP then in your sugar install go to modules/Users/authentication and place CASAuthenticate. I hard coded my server settings in so make sure to edit the files for your purposes. Edit – you can get a nicer version here on github.

Then edit config.php add

'authenticationClass' => 'CASAuthenticate',

to $sugar_config = array

SugarCRM is supposed to support SAML but I haven’t been able to make it work.

Google Web Store

Google web store is out, it’s just a bunch of links to websites. Really it does just about nothing. I love it! Why? Despite doing nothing at all, companies like the New York Times and NPR are making nice HTML 5, CSS3 websites. By targeting the “web store” they can ignore non standards compliant browsers. The sites are often much better and because they are just web links many of them run in Firefox and Opera. I did have some trouble loading the NPR app in Firefox though. Hopefully Firefox 4 and their web store will solve this. Anything that pushed companies to make nicer standards compliant websites is good in my opinion.

The web store does have one feature,  users can rate “apps” and the better apps can get featured.