What’s missing in Linux today

Seems like there’s always a new article about Linux rising to the top or failing. Year of the Linux Desktop, Linux has no market share and never will, etc, etc. Its seems that Linux both has solved every possible problem and has 110% market share while also having 0% market share and useless. Here are my thoughts on Linux being used to replace Windows in the enterprise based on my experience with it.

Wifi, still trouble (try using certain Ralink cards) but manageable and rarely a problem.

Office productivity is fine. OpenOffice LibreOffice (this name sucks but so does oracle so I’m divided) works. MS Office runs in wine but expect to pay for a commercial wine if you want multi user support. No serious issues here.

Printing, some just don’t work. But I’ve rarely encountered this. Many take some effort to work, but nothing beyond an entry level sys admin. Good enough for me.

LDAP support. If you’ve read this blog before you know there’s major issues here. It can be done. But with limitations. For example Windows lets you select your domain. You try and log in to a domain it waits for the domain to come up then you log in. You select local machine and it just does it without the need to wait. Linux….well nothing similar. I use Centrify Express but I have to choose between waiting (a long time at boot) for the domain connection to be made or just ignoring it and getting authentication errors (if credentials aren’t cached) when the computer is still connecting to the network and the domain. People expect this stuff to just work, since 2000 at least. A decade later and Linux can’t do it seamlessly.

File shares. Simple smb or nfs is easy yes. Want to put your home on nfs? Fine. WHO DOES THAT? It’s 2010 everyone uses laptops. People take the laptop outside and the home folder drops? This is not acceptable. I’m writing off NFS as useless on desktop computing. Windows deals with this with folder redirection. Your share folders act like mounted network drives until they are synced locally. When back online files get synced to the network. It just works as they say. I am looking into an old program called unison which seems to be able to sync folders. Also there is a program called sparkleshare that’s still in development that looks promising.  Both of these would give you Dropbox-esk sync functionality. That’s nice but still Windows does it better.

Application compatibility. This one probably puts off a lot of people. I actually think Linux is good here, of course I speak only from my own experience. My thinking today is where is not IS there a Linux version. There are notable exceptions that I simply don’t use. My biggest complaint here is that people need to make deb and rpm packages for their stuff. Seriously if you can afford to make some stupid custom install you can afford to make a package instead. Games are an exception but I’m talking about using Linux for enterprise desktops not personal.

More uno reports

I’ve been playing around more with openoffice.org’s uno api for making reports. Since I’ll be making more updates I’ll just post a link to the Google Code site

http://code.google.com/p/student-worker-relational-database/source/browse/#svn/trunk/ecwsp/uno_report

My latest improvement is supporting tables. It’s probably best to just show what it does.

Now a user could just download the report, change fonts, layout, etc, and reupload it. It does have it’s limitations and is a work in progress but it’s already pretty cool. As a developer I can just define what variables can be used and let someone else make the report (and change it around later). To use it you just have to make the proper data structures, so really this could be ported to any data driven application, not just Django.

LED lighting

I’ve recently purchased some LED lighting to see how it is. LED lights are more energy efficient and last much longer than CLF lights. That also don’t contain mercury and are very durable. I went with four of these

I have them installed in overhead recessed light fixtures. They seem to be ideal for this type of use because they are highly directional. The light seems to be on par with CLF but not as “warm” as incandescent. They work with a dimmer switch, though at certain levels they do flicker. It’s not a problem it happens at a predictable level that I just avoid. Pricewise they are very expensive. But I do expect them to pay for themselves in the long term in terms of energy savings and that they should last a very long time. Also I dropped one once and it didn’t break. I can’t figure out where they are made, they actually don’t say. The price should keep going down as more people buy them, but I’m ok with being an early adopter. Yeah energy efficiency!

Django and OpenOffice.org uno reports

Openoffice.org’s python uno library is great for report writing in a web application. I wanted to create a report template system where users can create templates with a word processor of their choice. I created a simple template model which consists of a CharField and FileField. Users can upload templates here in .doc, .odt, or any format OpenOffice.org supports.

Next I created some generic template report functions.


from django.http import HttpResponse
from django.core.servers.basehttp import FileWrapper

import uno
import os
import string
import tempfile

def findandreplace(document, search, find, replace):
"""This function searches and replaces. Create search, call function findFirst, and finally replace what we found."""
#What to search for
search.SearchString = unicode(find)
#search.SearchCaseSensitive = True
#search.SearchWords = True
found = document.findFirst( search )
if found:
print 'Found %s' % find
while found:
found.String = string.replace( found.String, unicode(find),unicode(replace))
found = document.findNext( found.End, search)

def replace_report(infile, outfile, data):
"""Replace words in a file use like this
data={}
data['$TEST']='worked yay'
returns a django HttpResponse of the file"""
# Boring uno stuff
local = uno.getComponentContext()
resolver = local.ServiceManager.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", local)
context = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
desktop = context.ServiceManager.createInstanceWithContext("com.sun.star.frame.Desktop", context)

# open document
document = desktop.loadComponentFromURL("file://" + str(infile) ,"_blank", 0, ())
cursor = document.Text.createTextCursor()

search = document.createSearchDescriptor()
#Do a loop of the data and replace the content.
for find,replace in data.items():
findandreplace(document,search,unicode(find),unicode(replace))
print find,replace

# create temporariy file to store document in
tmp = tempfile.NamedTemporaryFile()
document.storeToURL("file://" + str(tmp.name), ())
document.dispose()

# create http response out of temporariy file.
wrapper = FileWrapper(file(tmp.name))
response = HttpResponse(wrapper, content_type='application/odt')
response['Content-Length'] = os.path.getsize(tmp.name)
response['Content-Disposition'] = 'attachment; filename=' + outfile

return response

In this example I’m replacing text in a file. The result is returned as a http response. I can use it like this


template = Template.objects.get_or_create(name="My Selected Template")[0]
data={}
data['$date'] = str(date.today())
return replace_report(template.file.path, "file.odt", data)

In this case I am taking out my template and replacing all instances of the word “$date” with the current date. I choose to use $ to mark it as a variable but it could really be anything. Now when a user wants to change a font they can do it themselves.

Linux and Active Directory round 2

* edit Jan 12 2010. Better more reliable scripts.

* edit Oct 27th The netbook deployment went well this time. Feedback is generally that they have less problems than our Windows machines. Using Impress instead of MS powerpoint seems to be the biggest issue as Impress has trouble importing some powerpoint files in the office open XML format. Next on my todo list is play around with Unison file sync, NFS, and update this guide to reflect some improvements a colleague made.

The goal here is to create typical Windows Active Directory connect like Linux workstation. This includes being able to log in using Active Directory and mount various shared folders. It also must be idiot proof. We don’t want users saving on the desktop not realizing the desktop is not part of their smb share. It also must be cloneable. There should be absolutely no required interaction with the the computer after putting on this image.

Active Directory Integration

I choose to use Centrify for this. Likewise open is another option, but it seemed more buggy and I hate the way you have to configure it.

Ok make sure partner repositories are enabled and install centrifydc. If you want to make a clone-able image set your host name as something generic like “stockimage”. Add this script to crontab’s @restart

#!/bin/bash
hostCurrent=$(hostname)
hostOld='image'
commonauth='/etc/pam.d/common-auth'
if [ "$hostCurrent" == "$hostOld" ]
then
    (
    set -x
    host1=$(/usr/sbin/dmidecode | grep 'Serial Number: ' | sed 's/.*: \(.*\)/\1/;q')
    host2=
    host=$host1$host2
    host=$(echo $host | sed 's/[ ]*//g')
    hostname $host
    echo $host > /etc/hostname
    # TODO: axe this ugly hack and have upstart call us when we're connected
    counter=0
    while [ $counter -lt 60 ] && [ ! `/sbin/route -n | sed -rn 's/^0\.0\.0\.0[ ]+([0-9.]+)[ ]+0\.0\.0\.0.*/\1/p'` ]
    do
        sleep 1
        counter=`expr $counter + 1`
    done
    # Do NOT put regular administrator password here!
    # Use a special account and keep it DISABLED.
    /usr/sbin/adleave -u 'j' -p 'secret'
    /usr/share/centrifydc/bin/centrifydc stop
    /usr/sbin/adjoin -f -u 'j' -p 'secret' -w --name $host youradserver.com
set +x
    ) >& /opt/ad.log
    /sbin/reboot
elif [ "`sed 1q $commonauth | grep '^# lines inserted by Centrify'`" ]
then
    # Prevent double password prompt
    pammount=`sed -rn '/^auth[ ]+optional[ ]+pam_mount.so$/p' $commonauth`
    sed -ri '/^auth[ ]+optional[ ]+pam_mount.so$/d' $commonauth
    sed -ri 's/^(auth[ ]+sufficient[ ]+pam_centrifydc.so)$/\1 try_first_pass/' $commonauth
    sed -i  "1i\\$pammount" $commonauth
    /sbin/reboot
fi

That script will rename the hostname to something unique and join your domain. Because it’s on @reboot it runs every time the computer is turned on thus when you image it, it runs! Maybe put this in crontab last because you don’t want it running on your image. If you’re only joining one machine to AD then just run the adjoin command.

Ok bug work around time! I said Centrify is less buggy remember. This is only important for multi user machines, if there will only be one user you may skip this section. Ubuntu will boot well before you are online and thus a new user will get Authentication Error when trying to log in right away. To fix this first of all make sure if you use wireless that “enabled for all users” is checked in NetworkManager.

Next we need GDM to wait for centrify to connect before starting. We can do this with upstart.

Edit /etc/init/gdm.conf and look for the start on section. This basically tells GDM to only start once certain things have happened. Add centrify-connected to the bottom as seen here.

start on (filesystem
and started dbus
and (graphics-device-added fb0 PRIMARY_DEVICE_FOR_DISPLAY=1
or drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1
or stopped udevtrigger)
and centrify-connected)

Now it we need to make the centrify-connected signal. Edit /etc/init.d/centrifydc and look for case “$CMD” in start). Add “initctl emit centify-connected” under wait_adclient  Like this.

start)
adclient_check
echo -n "Starting $NAME: "
start-stop-daemon --start --quiet --exec $DAEMON --pidfile $PIDFILE \
-- $OPTIONS
RETVAL=$?
if [ $RETVAL -eq 0 ]; then
echo "OK"
wait_adclient
# upstart won't start gdm until we say we're connected
initctl emit centrify-connected  # added
else
echo "FAIL

That emits the signal telling GDM that Centrify is connected. The script is part of Centrify, it just didn’t emit the signal. It’s like they thought of this problem but didn’t actually fix it.You could reboot now if you wanted and it should work, but there’s much more we can do.

If your splash no longer shows up (happens to me 100% of the time on 10.04). You can try this command to fix it. I add this note because it can take a while for wifi to connect, then centrify to find AD, then GDM to start. If the users stares at the black screen for 1 minute they will probably assume the computer is just broken.

sudo -i
echo FRAMEBUFFER=y > /etc/initramfs-tools/conf.d/splash Code:
update-initramfs -u

Mount Windows Shares

Now your users will fire up nautilus and start browsing windows shares. smb://something.company.com/user$/myuser/documents is so easy to remember right? 😛

Lets face it users don’t know what a file share is usually. Lets mount them automatically for them like Windows does.

sudo apt-get install libpam-mount smbfs

Now edit /etc/security/pam_mount.conf.xml

make it looks something like this in the section <pam_mount> Make sure to change it for your own purposes. In this example I’m mounting a user’s documents folder.


<pam_mount>

<!-- Volume definitions -->

<volume user="*" fstype="cifs" server="server" path="users/%(DOMAIN_USER)" mountpoint="~/Documents" />

Don’t try mounting anything as Desktop because gnome won’t like it. There is a workaround here if you really need Desktop to be part of the share. First turn off the auto-creation of these folders (this puts you in charge of making them! Nautilus will not start if it doesn’t have these folders! Yes Nautilus sucks.) Edit /etc/xdg/user-dirs.conf and set enabled to False. Now edit /etc/skel/.profile and add the lines

mkdir "$HOME/Documents/Desktop" 2> /dev/null
ln -s "$HOME/Documents/Desktop" "$HOME/Desktop" 2> /dev/null
mkdir "$HOME/Documents/Downloads" 2> /dev/null
ln -s "$HOME/Documents/Downloads" "$HOME/Downloads" 2> /dev/null

This would symlink the Desktop and Downloads folder to be inside of Documents. Documents in actually a smb share. This should keep users saving files in places that are in smb.

There are two Ubuntu bugs related to smbfs. If your system hangs at logout look here

https://bugs.launchpad.net/ubuntu/+source/libpam-mount/+bug/574329

If it hangs at shutdown look here

https://bugs.launchpad.net/ubuntu/+source/samba/+bug/211631

Ok another Centrify bug work around is explained here Centrify goofs up pam_mount. I mean who would really want to connect to shares AND use active directory  😉  If you can’t handle clicking

1) Open /etc/pam.d/common-auth file

2) Check if “auth optional pam_mount.so” is the first line and “auth sufficient pam_centrifydc.so” second line. If “yes” then change the second line to:

auth       sufficient     pam_centrifydc.so try_first_pass

I also found that adding try_first_pass to the pam_unix.so line will allow non AD users to log in without entering the password twice.

Odds and ends

If you want to give some users sudo edit /etc/sudoers and add

%ADMIN\\UnixAdmins ALL = (ALL) ALL

That command would give the UnixAdmins group in the domain ADMIN sudo access.

If you want to edit the default desktop install a program called sabayon. It crashes sometimes but when not crashing it works pretty well. For some reason rebooting fixes it’s crashing…weird.

Another problem is that non admin users can disable network manager for the entire system even after a reboot! I’m not sure what to do about this, it’s a major headache because users won’t be able to log in at all without networking. I can’t think of any fix other than disabling network manager, but this is not ideal as sometimes there are legitimate uses for network manager.

As Mike pointed one could use the adjoin –selfserve command if AD knows the hostnames in advance. I choose to use an account that is usually disabled so if users see the password, they won’t be able to do much.

Compared to Windows this is a huge pain the first time to get right but cloning is way easier. I love running one image on many different models of computers. I moved the image around to several computers fine tuning it for each one. That included installing specific drivers. In the end I have one image that can be deployed anywhere.

I’m still missing a few must have features such as syncing the entire home folder to a share. NFS home folder is not enough, try leaving the building with your NFS home folder laptop. iFolder is an option I want to look into more. My initial experience is that it’s difficult to configure. It also doesn’t give you a CIFS interface which is often nice to have. There are any number of hosted solutions (Dropbox, JungleDisk, Ubuntu one, etc) that might work for you. It’s a shame Canonical won’t offer this as an on site solution like they do Landscape. Sparkle Share is an upcoming project that aims to meet this need but it’s not ready yet. You could also look into CMS software such as Alfresco.

Creating a custom Django DateTimeField model

In my last post I ran into a problem. A legacy database stores values as BigInt in MySQL using Unix time. Django uses datetime. I want the nice validation and widgets from Django’s DateTimeField. Here’s how to solve in.

Create a new field, I called it UnixTimestampField and overrode models.DateTimeField.

from django.db import models
from datetime import datetime
from time import strftime, mktime
import time

class UnixTimestampField(models.DateTimeField):
__metaclass__ = models.SubfieldBase

def __init__(self, null=False, blank=False, **kwargs):
super(UnixTimestampField, self).__init__(**kwargs)

def db_type(self):
typ=['bigint']
if self.isnull:
typ += ['NULL']
return ' '.join(typ)

def to_python(self, value):
super(UnixTimestampField, self)
try:
return datetime.fromtimestamp(float(value))
except:
return value

def get_db_prep_value(self, value):
if value==None:
return None
return time.mktime(value.timetuple())

def get_prep_value(self, value):
if value==None:
return None
return time.mktime(value.timetuple())

It seems to work. I’ll update though if I run into any trouble down the line or re-factor it. This is my first attempt at making a custom Django Model Field.

Django models from non Django applications

I have an existing LAMP application that I want to write some quick code for. I want to easily manipulate the database without taking the time to understand how the application works fully and mess around with PHP. The solution, create a Django model to act as a wrapper for the sections of the database I need.

First I created some very simple views in MySQL to my new Django created database. I could have just put both programs in the same database but this should make updates easier. Remember in MySQL views are updatable only when the view references one table. All my views look like
create otherdb.my_view as select * from newdb.some_table;
With this method I can easily drop and replace the other database when I update it.

Next I run Django’s inspectdb command to get some generic Django model code. Edit this to one’s liking. For me that meant renaming fields, deleting the primary key (so Django automatically makes it. Only works if it is in fact a primary key auto incr integer). Turn foreign keys into foreign key fields. Make dates into DateFields.

Setup my Django admin interface and all the sudden I can use the spiffy Django admin interface on a completely different application. YAY!

However I ran into a small problem. I’ll post this separately for people who only care about Creating a custom Django DateTimeField

Recipe

I’ve decided to vary things up and post a recipe I made up, expect more to come as well as the normal mostly Django and Linux related posts.

Amazing Vegetarian Sandwich

  • Tofu
  • Fresh veggies (tomato, peppers, anything)
  • Bread
  • Curry
  • Flour
  • Cheese
  • Butter or Oil

Usually non meat sandwich’s suck and aren’t filling, but this sandwich is bad ass and will make the meat eaters jealous. First dry the Tofu the best you can. Slice it, powder it with flour, then fry it in butter or oil as fast and hot as you can. Sprinkle some curry on it while it cooks. Fry up some veggies if desired too or put them on uncooked. Finally add some Italian dressing or olive oil and pepper. Swiss cheese is good for sandwiches but any kind would work. These sandwiches are great for a picnic and bringing leftovers to work the next day.