How to actually upgrade Joomla 1.5 to 2.5

Joomla 1.5 has no upgrade path to 2.5. If you are still using 1.5 you need to upgrade or you will get hacked. It’s only a matter of when not if. However their upgrade path is utter garbage. It’s broken 10 times over and absurd. Here’s how.

  1. Enable System – Mootools Upgrade in Plugin Manager
  2. Download JUpgrade, except the download page sucks and took me a few hours to figure out I had to “add to cart” the free plugin. Did you know Mega is a site you can host files on? Install the plugin when you have it.
  3. Click components, JUpgrade
  4. Don’t even try upgrading it won’t work and the plugin is totally broken. Click Parameters and skip the checks and download. Save.
  5. Copy the latest Joomla 2.5 (a zip file) into /your_joomla/tmp and make sure it’s named exactly joomla25.zip
  6. Now start the upgrade. Upgrade is a bad name more like copy some of your pages maybe if I feel like it. It’s a partial migration tool at best that I assume was made by the WordPress team to encourage users to switch.
  7. Fix the 1000 problems you’ve introduced. Don’t worry it makes a copy for joomla 2.5, your 1.5 site is still fine for now (till it gets hacked any day now). You can start here.
  8. Migrate to wordpress when you have more time because any product that has no upgrade path sucks. WordPress is a one click upgrade and I’ve never seen it break templates (YMMV).

Puppet – managing desktop computers with awesome/sucky tools

Puppet is a configuration management tool. It can be used to manage desktop clients. You could compare it to Active Directory GPO, sort of. Instead of running bash scripts or whatever you tell Puppet what you want the clients to look like and it will attempt to do so. For example you can say google-chrome should be up to date and if the client is not up to date it will take steps to do so (apt-get). Like many open source projects I think it’s much harder to learn but more powerful than say GPOScreenshot from 2013-01-24 15:20:53Puppet is not so straight forward to learn and use. Why do products think I give a damn to learn about them? I don’t. I’m not a puppet developer. I don’t care about configuration management. My users sure don’t care. This post should hopefully tell you how to get started with puppet and push changes to clients. Here is a systems chart of what I want.

Screenshot from 2013-01-24 15:53:01

Server Installation

I’ll assume you want a puppet master server and clients. To install puppet master: http://docs.puppetlabs.com/guides/installation.html#debian-and-ubuntu

Not so bad. puppet-dashboard is the gui you see in that browser of my screen shot. It lets you see how clients are doing if they are checking in, making changes, MIA. It also lets you make groups in a gui. Installation is confusing because ruby on rails is crap in Ubuntu if you aren’t familiar with the process.
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html

Since you want to push changes to clients and not learn ruby on rails garbage make sure to use the lame and boring webrick stuff that comes with the ubuntu packages you install. The documentation doesn’t really explain this at all. It says how to run it over command line for testing and a link to some ruby crap. Awesome. Here’s an idea – why not link to programming in ruby and tell the user to make their own puppet? Edit /etc/default/puppet-dashboard and /etc/default/puppet-dashboard-workers and set START=yes so that the processes start. You don’t care how it works unless you are running some huge enterprise and want to use apache instead. Now you can start it (once you followed the above link and set up databases, etc) by running service puppet-dashboard start. I got some permission errors at first that a chmod -R 777 took care of. Who cares. Guess the Ubuntu packages are buggy.

Dashboard has some inventory feature too, just ignore it. I did get it to work after many hours and it’s not terribly useful and took a long time to do.

Client Installation

Again ubuntu 12.04

  1. apt-get install puppet
  2. edit /etc/default/puppet and set START=yes
  3. edit /etc/puppet.conf and blindly add
    server = yourpuppet.server.com
    report=true
    pluginsync=true
  4. sudo puppet agent –test
  5. On puppet master run
    puppet cert sign –all
  6. If you want run on client that test again to see if it works. You should see it in dashboard if that was set up right. If not make sure the workers are running and the report stuff is set right
  7. (optional) add your “node” which is a client to a group and class. If you add it to a group you can add the group to a class. I don’t understand the difference actually.

Now to push a change. I found this a good reference because it’s task oriented. You can push things to a certain group (like maybe a testing group?) like this.

class testers {
do stuff!
}

You can set up the nodes in dashboard as I said make sure the group is in the class though. The language is ok but there are some stupid limitations. I found it impossible to run apt-get update before installing a new package which seems a reasonable thing to do. The solution on stackoverflow will make apt-get update run EVERY time which means in dashboard everything will report it’s making changes every time. I’d like to know the difference between a real change (I installed a new package) vs a stupid one (I ran apt-get update because I suck and have to). I just solved it by adding a cron job to apt-get update every day, meh. Adding the cron job file counts as a change but then cron runs itself and that isn’t a puppet change.

Once you figure it out, puppet is very nice. It’s better than GPO because Windows sucks and has no package management and no decent scripting language to do stuff without a gui. Most stuff doesn’t have a msi file so GPO can’t be used. In linux, most applications have a repository or at least a .deb file.  The ability to make system wide changes on the fly is wonderful. Tasks that would have taken too long are suddenly possible. Here is my site.pp file in you want to see what I use it for and complain about my terrible hacks.

Review of Ulteo Open Virtual Desktop

Ulteo is a open source, citrix like, way to run remote applications. For my purposes I wanted Linux and Windows 7/8 users to run legacy and Windows only applications without me having to install things that require Internet Explorer 6 for instance.

The simplest route would be just using remote desktop on a windows terminal services server. This can be a bit annoying however having to log into another computer just to access some application. Ulteo can make applications run more like they are local and be launched via a web portal.

Yep, that’s IE on the Unity bar. And no it doesn’t support quicklists.

Installation

Ulteo is a huge pain to set up. It took me many attempts. My requirements were that it authenticated against Active Directory and get all the mounts set up as usual. I set up ulteo on a virtualized Linux host and installed the windows application server in Windows 2003. By default Ulteo creates lots of profiles in Windows (and Linux) application servers as users log in, this lets it work without Active Directory. However I want to use Active Directory because I need it to set up all the typical letter drive mounts and any other GPO stuff. There is an option for this but it breaks compatibility with using a Linux application server. I don’t really mind this but it’s annoying not having the option to do both.

I also spent a good amount of time figuring out how to get users to only see the web portal. The default option gives you a full desktop, which is not what I want. I’d just use RDP for that. Some configuration options seems to have no effect at all. It all feels a bit unpolished for something that end users are going to depend on day to day. But hey it is free. They of course offer paid support but not having any budget rules that out.

I also tried setting up a new ulteo server with a Linux ldap backend and Windows App server. After many hours I just gave up. I don’t see any way to do this. It seems like the non AD user profiles system is broken.

User Experience

It’s not bad. I don’t praise things much more than that. It’s very slick to sign into a website and launch applications that appear to be running locally….well mostly. It’s using seamless RDP to do this and it can flake out. It’s possible to say launch explorer and ruin the experience. But most users won’t be looking for that sort of thing.

It can map local drives to the remote server which is useful. Sometimes though it can be a bit much for users who can’t handle much thought outside of “I better save in the P drive” but if GPO is set up their “P” drive should be there.

I did find Unity/Gnome panel/whatever doesn’t always understand what’s going on. Many applications launch multiple icons but clicking on some won’t do anything. For instance if the application is trying to put something on the Windows toolbar, linux isn’t going to know what it is. This can cause some pain for user clicking on things that do nothing and having to search for the real window. Another issue is that pop up menu’s don’t always behave. A date picker widget closes before it can be clicked. A menu appears at the top left of the screen instead of where it should.

It all uses Java which is probably already installed, unless Canonical decides they want to forcibly remove it from your computer. Yea I’m still not happy about that incident.

I have real end users on it and no complaints with reliability. In my testing I found I was able to get in a state where the session crashes and then I can’t log in. It’s very dumb, why can’t I kill my old session and log in? Why is that not an option!? I’m sorry you already have a session now go to hell. But it hasn’t come up with end users…so far.

Overall I’ve successfully deployed Ulteo. It works. Citrix is prohibitively expensive (and is also crap anyway, could you make it any damn harder to download the linux/osx ica client?). I won’t have to maintain older Windows XP hold out computers for legacy crap. I won’t have to install annoying applications if I don’t want to locally. That saves me time. User can potentially access on site applications at home without messing with rdp. Citrix has a worthy competitor and it costs nothing. I would guess for most people however Ulteo will be too hard to set up. Most configurations I’ve tried just don’t work at all.

Buying new computers is pointless

This post doesn’t apply if you’re rich. In that case enjoy your ivy bridge i7 with Terabyte solid state drive you jerk.

At one work site they still use Windows XP. That’s isn’t going to fly next school year when support ends. Time to upgrade. So new computers? Hell no.

You brand new computers sucks and my old x61 will outperform it

New mid range computers suck. For a typical office worker cpu speed and memory are useless. The only bottle neck is hard drive read speed but still only high end computers use them. If using Windows you need 4GB+ ram. Above that who cares.

Experiment

I want to see if upgrading old 2008 computers can out perform a new computers in the only test that matters – user perception. Let’s compare costs.

Upgrade costs

  • Windows 8 $75 (I know this varies a lot)
    • or Ubuntu 12.04 (free)
  • 128GB SSD $100
  • 2 GB stick of RAM $30

That’s $130 if using nice software or $205 if you like interfaces that are fucking stupid.

New costs

What the hell – everything sucks. The cheapest Windows computer at the time of writing on Newegg with a SSD is $1010. Damn. And that’s with sandy bridge (not the latest cpu) and crap resolution. The best (value) computer I could find was a chrome book. I guess you could install Ubuntu on it. You’re better off buying a older Thinkpad t400 and putting in a solid state drive yourself. I did the math – $700 at the time of writing for a refurb t400 with SSD. Of note on System76 you can get a ivy bridge lemu4 with SSD for $709. Not bad if you don’t like installing hard drives.

So $700 vs $205. Hmm. But a new computer might be faster? HAH. Maybe if your company’s core product is calculating pi all day. Windows 8 or Ubuntu I’ve had nothing but gasp on how fast a computer is with a solid state drive. People wonder why their new home computer is so slow.

So stop waving your big cpu L1 cache size around and just get a SSD.

Thoughts on Zentyal

* Update Jan 2013 – Zentyal 3.0 has been out and a migration guide published. You can’t migrate user passwords which for effective purposes means you can’t migrate. IMO this is unacceptable and I would not suggest using Zentyal unless you are able to get very dirty with ldap schema and samba to migrate them yourselves. If you set up clients using their old desktop package you are just thrown under a bus because this configuration won’t work with 3.0 without MAJOR hacks. Can you imagine Microsoft releasing a new version of Active Directory where you had to redo every client and have every user set their password? No you can’t because they wouldn’t ever do that.

A few times people have suggested I try out Zentyal. So here it is, my review of Zentyal 2.2.

One of my biggest complains with Linux in the enterprise is that there are 100 different solutions and you need 30 of them and all of them suck. If you want centralized login, file shares, email, etc in Windows you get Windows Server. It does everything you need. Joining a domain takes about 20 seconds. In Linux you better be prepared to spend years learning every little part. Where do you even begin? OpenLDAP? FreeIPA. Who the hell cares what LDAP even is.

Zentyal solves this. Want users and groups? Well install the module users and groups. Done. Zentyal picks all the choices for you. You don’t want Samba and you sure don’t give a damn about how it compares with NFS. You want users to share files. So install the “file sharing” module. I really like this screen shot, it speaks to me for all the times I see “LDAP config” in whatever application. Here this is probably what you are looking for just copy and paste it.

In the backend Zentyal just generally uses whatever is most popular. OpenLDAP, Samba,  Squid, etc. So if you want to fight with LDIF files you can. What’s sad is you will have to 🙁

Zentyal has some glaring flaws. You can’t even import users. It’s light years behind Active Directory in terms of features – though I do prefer having a web application over Active Directory tools. Also you can run Zentyal on some old junk you have laying around, Active Directory basically requires a super computer these days. You can of course use scripts but still it’s disappointing.

Zentyal is written in perl which is about the most angry language ever made.

perl -pe '$_.="\n"x7'

That’s perl for who the hell can read this. Sorry to be picky but being written in perl means I won’t contribute. And I might otherwise so that’s too bad. Would be great to have something like a inventory script in Zentyal.

Another issue that really bugs me is the total lack of consideration for clients. They have a Linux desktop debian package but it’s outdated and actually does very little. It will make your ldap.conf file for you and that’s nice. But forget cached credentials. I’m sorry a solution that doesn’t work on a laptop is just absurd. Of course you can set it up yourself but that’s a pain. They also have a unison script (just like what I did) to simulate roaming profiles, but I don’t really like their implementation. It adds scary windows to a user’s log in and doesn’t sync often enough to be reliable.

All in all I’m happy I migrated to Zentyal from vanilla openldap, samba, etc. It’s a step in the right direction. I was even able to integrate Zentyal with Google Apps (this isn’t supported, big hack). I’ll try to get around to sharing all the hacks I did to make password syncing work soon. Stay tuned.

SMART notebook bug breaks Ubuntu gvfs

I found a nasty bug in SMART notebook for Linux. The particular package nwfermi, which is a driver to support 400 series boards, will change your /dev/ permissions. This happened in previous versions of Ubuntu but is much more noticeable in 12.04. It stops gvfs from working.

If you want to reproduce the bug just install the nwfermi package from SMART then ls -la /dev/fuse. You will find it no longer has the “other” permissions. If you mount a samba share via nautilus it will work but not through gvfs. That makes it rather slow and won’t give you a mounted path (~/.gvfs/your_share). This in turn will make the share not work with programs that don’t support a smb file path such as wine.

The solution is to just not to install nwfermi. I had it installed just because I didn’t think it would hurt anything. Well it will.

Setting “Desktop” to be ~/Documents/Desktop

xdg is a unix thing that tells you where “important” folders are. Such as Documents, Music, Desktop, etc.
A simple (and documented example in the Ubuntu conf file) would be to have subfolders. To emulate windows I could say that Desktop=Documents/Desktop in /etc/xdg/user-dirs.defaults
However if Documents or Desktop is a mount point bad things happen. It just defaults Desktop=Home. Actually it does work randomly…a race condition between pam_mount and xdg I suppose. Not that xdg creates folders if they don’t exists…at least it usually does. So the issue isn’t that the folder doesn’t exists. Just look in /etc/skel there is no Desktop folder.
There is a command xdg-user-dirs-update that can reset these if you pass –force. It works! Run it, go to nautilus, click desktop and it shows you Documents/Desktop in the mount point! However it does not refresh nautilus desktop so the actual desktop still points to Home. One could fix it manually in this terrible hack.
Edit ~/.profile or /etc/skel/.profile for new users too, add to it
/opt/xdgfix.sh&
Now create a file /opt/xdgfix.sh and make it
sleep 6
xdg-user-dirs-update --force
killall nautilus
nautilus -n&

That’s sleep 6 seconds, update our xdg user dirs, stop nautilus, start nautilus.

So what good is xdg if it doesn’t work reliably? It seems to have totally erratic behavior in each version of Ubuntu. Before 12.04 it just wouldn’t ever work with mount points. Now we have sometimes kinda works….some progress 🙁

Of course this is just one problem out of many that stem for Linux having nothing even remotely close to Windows Folder Redirection which if you haven’t guessed is what I’m trying to painstakingly emulate.

Ubuntu 12.04 deployment with Active Directory

** Update 4/13 – Added winbind. Winbind is now my preferred solution.

This is a follow up to my past post. I want to deploy 12.04 for what I consider a typical enterprise environment. That means centralized authentication, file shares, roaming profiles, etc. Your first step should be to acknowledge this is a very hard project to do. If you need to deploy some computers this week, go get your Windows install CD right now. Linux takes about 10 100 times as long to set up.

Authentication to Active Directory

Options include Centrify, likewise open, winbind, and pam_ldap. They are suck terribly. My opinion on which is the the best differs often. Right now I’d say winbind. If you just need a quick solution just for your own use, likewise works and is much easier. Here’s a quick comparison:

Winbind Lightweight and fast. Terrible documentation. Hard to set up.
Likewise open Easy and includes a GUI. Causes slow log in times.
Centrify Very buggy, would not recommend.
pam_ldap Both buggy and hard and lacks features.

There is also sssd which I have never been able to get working. You can read this post in the Ubuntu-Enterprise list server.

Either way they both suffer from a fatal flaw that would probably keep all but the most dedicated linux sysadmins far far away from Linux. They don’t work in a real networked environment! I’ve talked about this issue before, basically lightdm starts up before networking is up in a race condition. The user types in a name, password, gets an error, calls tech support and has the impression that this “Ubuntu” is quite the useless operating system.

First with either option you need to allow users to type in a login in lightdm. Edit /etc/lightdm/lightdm.conf and make it so users can type in a username by adding

greeter-hide-users=true

Also make sure your wireless network is available to all users. (Click the network icon, Edit Connections…). Next follow instructions for either Likewise or Centrify.

Likewise Open

It’s terribly easy to install it. It even has a GUI. So I won’t include instructions for this. However if you need it working on boot as I described above do this. Place this script somewhere like /opt and make it executable. It just checks to see if any networking is up or time out at 20 seconds. You may need to adjust it or even place a sleep statement after it. Just experiment on your hardware/network.

#!/bin/bash
i=0
while [ $i -lt 20 ]; do
 sleep 1
 is_up=$(ping -q -w 1 -c 1 `ip r | grep default | cut -d ' ' -f 3` > /dev/null && echo 1 || echo 0)
 let i=$i+1
 if [ $is_up -eq 1 ]; then
 let i=999
 fi
done

Now edit /etc/init/lightdm.conf and add under the emits section. This script just runs ping test before starting lightdm then waits 5 extra seconds to give things time to settle. You should test this our throughly and make it longer if you have authentication issues.

pre-start script
 /opt/pingtest.sh
 sleep 5
end script

winbind

Ubuntu community wiki will give you information that may or may not be vaguely related to a bunch of terms you may or may not understand. Let’s start with some definitions to be clear what we are doing.

Kerberos – A system that let’s users log into services like share drives without retyping the password. Basically it’s some crap you need to set up.

win_bind – This does the actual authentication. It’s part of Samba.

Install these packages

sudo apt-get install winbind samba smbfs smbclient krb5-user

The krb5 installation will ask you for a realm. If your domain is ADMIN and your network is something.com then your realm is ADMIN.SOMETHING.COM. I didn’t have to tweak the krb5 conf file really. Test it with knit some_username. If it doesn’t work experiment with re-configuring it using  dpkg-reconfigure krb5-user. I didn’t do this stuff at all and don’t understand what it is.

Next set up samba by editing /etc/samba/smb.conf

Here is my smb.conf just replace the domain with your own. Who knows what all that does. You can read about it more here. Test your smb.conf by running sudo testparm

To “Join the domain” run

sudo net ads join -U Administrator

You will get a DNS update failed but that’s just a trick to make you think you screwed up the configuration.

If you want users to be able to change their passwords, check out this bug.

Do you like having home folders? Well the default now is not to create them. That sure makes sense. Add them back.

Now run sudo pam-auth-update to update pam configuration. I strongly suggest ensuring pam-auth-update works and that you don’t manually hack the pam.d files. On an update your users might unknowing blow away your hacks.

Test the set up but be aware it probably won’t work on boot until you log in as someone else. This is because of the network race condition issue. See the part about pingtest in the likewise open section. Do the same thing. I also discovered that when winbind is in the unworkable state where you can’t log in, running wbinfo -u fixes it. So I made my lightdm init script like this instead. Yay hacks.

pre-start script
 /opt/pingtest.sh
 /usr/bin/wbinfo -u&
end script

If you have been toying with other authentication mechanisms your /etc/nsswitch.conf might be off. See here. nsswitch stands for Name Service Satanic Witch – who wants to curse you. You must satisfy the witch in order to proceed.

Centrify

In my experience in July 1012 Centrify will cause odd networking issues. So I don’t suggest it. If you want to try anyway:
You can get it from the Ubuntu partner repo’s. If you don’t have this, you need to check this off in Ubuntu Software Center, Edit, Software Sources. Then update then install centrifydc. Join your domain like

adjoin -w mydomain.org

Centrify doesn’t play nice with pam-auth-update so create a conf file for it called /usr/share/pam-configs/centrifydc

Name: Centrify DC
Default: yes
Priority: 257
Auth-Type: Primary
Auth:
        [success=end default=ignore]                            pam_centrifydc.so try_first_pass
Account-Type: Primary
Account:
        [success=end new_authtok_reqd=done default=ignore]      pam_centrifydc.so
Session-Type: Additional
Session:
        required                                                pam_centrifydc.so homedir
Password-Type: Primary
Password:
        [success=end new_authtok_reqd=done ignore=ignore default=die]   pam_centrifydc.so try_first_pass
Password-Initial:
        [success=end new_authtok_reqd=done ignore=ignore default=die]   pam_centrifydc.so

Now run pam-update-auth and centrify will play nice with other modules such as pam-mount

Next we need to resolve the can’t log in on first boot problem. In likewise we just made a simple is networking up yet test. This doesn’t work with Centrify. From what I can tell this is what happens.

  1. Networking starts
  2. pingtest script launches lightdm only after networking is up. Right now we could ping the domain controller
  3. Centrify attempts to contact the domain controller but fails for some unknown reason. It then sites and does nothing for a bit. Then tries again and succeeds. This takes about 30 seconds.
Instead we need to modify /etc/init.d/centrify and add some upstart emits. These will tell us when centrify is ready to use. Look in the start section and add this emit as shown in bold
start)
 adclient_check
 echo -n "Starting $NAME: "
 start-stop-daemon --start --quiet --exec $DAEMON --pidfile $PIDFILE \
 -- $OPTIONS
 RETVAL=$?
 if [ $RETVAL -eq 0 ]; then
 echo "OK"
 wait_adclient
 # upstart won't start gdm until we say we're connected
 initctl emit centrify-connected # added
 else
 echo "FAIL
This emit signal doesn’t do anything in itself. It’s just an upstart signal we can look for else where, such as /etc/init/lightdm.conf where we will add it at a dependency to start.
start on (filesystem
 and started dbus
 and (graphics-device-added fb0 PRIMARY_DEVICE_FOR_DISPLAY=1
 or drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1
 or stopped udevtrigger)
 and centrify-connected)
stop on runlevel [016]
But what if the user is at home and will never connect to the domain controller? Well it will take a long time to boot. I’m sorry. If you have a better solution please comment. Let’s compare are boot time to see if it’s reasonable. All tests are on a Lenovo x120e. They are from power button to login prompt
Ubuntu 12.04 with my modified init scripts.
53 seconds (28 of those seconds are just waiting idle for Centrify)
Windows 7
47 seconds
Windows XP
A blazing 26 seconds! It would be pretty hard to recommend Linux if MS didn’t screw up NT with Windows Vista/7.
Anyway I’ve determined we are almost as fast as Win 7 which is what a x120e comes with. While it makes me cringe that I’m making the computer even slower, it’s on par and that makes it a good enough solution. Factor in time waiting for Win 7 to be generally slow after login and one could argue Linux is just as….slow. As much as I hate Linux, I hate Windows more.

Wifi for non admin users

The default in Ubuntu is that this can only be done by admin users. Linus Travolds himself actually ranted about this policy in opensuse. Here is the fix.

Network users can’t add printers

By default a network user won’t even be able to add a printer. Add all users to the lpadmin group using this.

Make updates not replace conf files

By default Ubuntu asks users if they want to replace configuration files on updates. A more honest dialog would say please click a button, one will break your system while the other lets the update continue.

To disable this – create the file /etc/apt/apt.conf.d/local

Dpkg::Options {
 "--force-confdef";
 "--force-confold";
}

An addition step would be to enable unattended upgrades. This will force most updates without user intervention. That scares me a little. Here are instructions on enabling it. Notice the option to break down upgrades into small chunks so that shutdown is still possible without too much delay. No Windows style 30 minute shutdowns hurray!
https://help.ubuntu.com/community/AutomaticSecurityUpdates

Desktop Environment

People seem to like Unity these days, but I still don’t. It’s also unfamiliar with new users. I’m more concerned about reducing tech support calls than wowing users. It’s fairly easy to remove it (but why no gui way?). Install gnome-fallback-session. Now edit /etc/lightdm/lightdm.conf and set

user-session=gnome-classic

Log in as a new user to test, to ensure it’s not just using the previous session. I like to just have one bottom gnome-panel sort of like classic windows.

Also install compiz config manager and enable alt tab which is notably missing! And once more I suggest playing around with compiz on your computer model. It’s often can be very buggy and you may want to just fully disable it.

Copy your ~/.config file into /etc/skel but then delete anything you don’t want (firefox, chrome, etc) so that new users get the same configuration you have now. There used to be a program called sabayon that did this in a more user friendly way but it’s too buggy and it’s faster to just copy things to /etc/skel. All in all the Windows way of having All Users is much easier IMO.

Also for some reason I don’t quite understand you need to follow this to make /etc/skel copy files to the new users home. What I don’t get is that they seem to copy on a vanilla ubuntu client but as soon as you set up pam_mount they just stop working without this. Anyway copy the file and run pam-auth-update and it works.

Centralized Printing

You can control printing by having a cups server.  Run system-config-printer and click Server Settings. Check Show printer shared by other systems. Hit Ok. Now Click Server settings again (this is a bug). Now advanced. Now click add and type in the address of your cups server. The printers will just show up like magic! If users aren’t allowed it to print it just won’t print. Which is really annoying actually. Why can’t it prompt for a password? It used to try this but the prompt was broken and never actually worked 🙁

I submitted a bug report about it not asking for credentials.

I’m tempted to try out samba4 printing for Windows clients. In my experience it does a poor job except in trivial configurations. I wasn’t able to list printers in the directory for example.

Windows Applications

There’s probably a lot of Windows only applications you need to run. Crossover/Wine work for some like Office 2010. It’s really really buggy though. See my experience here. For programs that won’t run in wine you can use rdp. Ulteo makes a cool platform that lets users launch apps from a website. Great for Internet Explorer and centralized systems like proprietary databases. However it’s a pain to set up, so be prepared!

Java

Update – I find this the easiest method.

In short – It’s just about impossible to install Java now. Really I have no perfect work around. Maybe you can get away with OpenJDK which runs about 0 out of 10 applications I use. Canonical might just take away your java without warning. You could install it from Oracle…but your users won’t get updates this way. You could try various PPA scripts but it seems Oracle is actively trying to stop this. With the Oracle installation you are left wide open to exploits without updates. Lovely. You’re an IT person if you’re reading this. Do me a favor and DON’T EVER BUY ANYTHING FROM ORACLE.

File Shares – Samba

Samba works with the gui but you still might prefer to use pam_mount. I already talked about it in my previous post so I won’t again. It’s better if you want shares to come up automatically for users. If you just have a few users they can use nautilus to find the shares. Just type in smb://yourserver/share and save it as a bookmark. There is a browse share feature, but I’ve never seen this work in my life. But beware the bookmark thing won’t appear in wine! So if you need MS Office you users won’t be able to save as into the share!

If you do use pam_mount make sure to work around this bug that prevents likewise-open from working with it.

Update – pam-mount with a fast computer bug – On a modern computer a user might be able to log in before networking is up. If you use the pingtest.sh I described above you won’t be affected. However if the user logs in before networking is up, shares won’t mount. Obviously pam-mount will give up instead of retrying because it’s utter garbage and a better name might be pam-do-nothing.

Rant – Mounting a file share sounds like the most basic enterprise function, but Linux has no good solution. The GUI way doesn’t work in all applications. Pam mount is not ideal because if you turn on the computer without networking (at home?) it requires a log out (or terminal su) for it to work. Also pam_mount rarely umounts correctly which can cause odd delays sometimes. It just sucks.

Imaging

I like to use clonezilla for imaging. Here is a script you can use to automate AD joining. Take a minute to look this over and put in your info. Then run it on cron @reboot. It runs only if it sees the hostname as image. Note this script is for likewise open. For centrify just replace the join command.

#!/bin/bash
hostCurrent=$(hostname)
hostOld='image'
if [ "$hostCurrent" == "$hostOld" ]
then
 date > /opt/ad.log # overwrite log the first time around
 # DOE mangled machines to the point that the first serial number is blank!
 host1=$(/usr/sbin/dmidecode | /bin/grep -E 'Serial Number:[[:space:]]*[^[:space:]]+' | /bin/sed 's/.*: \(.*\)/\1/;q')
 host=$(echo $host | /bin/sed 's/[ ]*//g')
 hostname $host
 echo $host > /etc/hostname
 /opt/pingtest.sh
 sleep 10
 (
 /usr/bin/domainjoin-cli join your.domain.org 'user' 'password'
 adreturn=$?
 ) 2>&1 >> /opt/ad.log
 if [ $adreturn -ne 0 ]
 then
 echo "adjoin failed." >> /opt/ad.log
 exit 1
 fi
 echo "I'm itching to reboot." >> /opt/ad.log
 /sbin/reboot
fi

I made this to take inventory. Sorry it sort of sucks and has no installer. Basically the idea to that a script runs that feeds some stats like hostname and hd freespace to some database.

Admin access

If you want to give some users sudo edit /etc/sudoers and add something like.

%ADMIN\\UnixAdmins ALL = (ALL) ALL

But note that there is a bug that prevents it from really being useful. This will let users do sudo on the command line but not in the gui. The work around is to just manually add each user to the local sudo group. Have fun with that. Comment if you have a better way.

File Sync (roaming profiles)

You can try using unison. Owncloud has a desktop sync tool but it’s still in beta sucks. I’m using that unison script for now. For desktops just use samba instead since you don’t need the files to copy to the local hard drive.

Web Browser

I use Chrome as the default web browser. There are some not so well documented ways you can make it a nicer experience with default preferences. I really hate the many pop ups it has on first launch for instance. First read this
http://www.chromium.org/administrators/configuring-other-preferences
I don’t think it does a good job explaining that master_preferences goes next to the google-chrome executable (i.e. /opt/google/chrome/master_preferences)

The real exhaustive list of preferences is here right in the source:

http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/pref_names.cc?view=markup

‘user_skipped’ disables the sync promo for instance. Here is my master preferences file. Over 9000 promo views is just for fun.
{
 "homepage" : "companyportal.page.com",
 "homepage_is_newtabpage" : false,
 "browser" : {
 "show_home_button" : true,
 "check_default_browser" : false
 },
 "distribution" : {
 "skip_first_run_ui" : true,
 "show_welcome_page" : false,
 "make_chrome_default" : false
 },
 "first_run_tabs" : [
 "http://www.cristoreybrooklyn.org/portal"
 ],
 "sync_promo": {
 "startup_count": 1,
 "user_skipped": true,
 "view_count": 9001
 }
}

After install configuration changes

I use puppet, see this post. If you have time set this up before deploying you will save yourself a lot of time later. You might even consider placing a lot of the hacks I mention in puppet to ease future deployments.

Overall thoughts

12.04 IMO is as many steps forward as it is back. Some bugs are fixed and some are introduced. It’s a perfect example of why corporations don’t use linux outside IT. You can spend weeks setting up the perfect image in 11.10 but then 12.04 comes out and nothing you did before works. Here are my thoughts overall

+ Much better battery life
+ Gnome fall back session is a little less buggy.
– Gnome Classic is not as good as 10.04. For example the missing alt tab and in my experience compiz is less stable in Gnome Classic. Also sometimes wifi manager doesn’t display right for me. Again this is a trivial issue but for an end user it means calling tech support and for a company it means Linux is wasting money.
Likewise open is more buggy (doesn’t work with /etc/skel)
– CUPS is more buggy (no warning about authorization failure)
– Less configurable overall compared to gnome 2.
– Not Ubuntu’s fault, but java is now a nightmare

My advice to Ubuntu is to focus on quality control instead of UI. But it looks like they are going after consumers and not businesses. If they just fixed the bugs I mentioned I would recommend Linux to any small business. As is, I would only recommend it to companies that have a deep understanding of Linux already. Windows Server will continue to dominate until Ubuntu is Linux for human beings who have jobs.

As a LTS release 12.04 disappoints me. After all the configuration work – the end user gets a worse experience in my case. A longer boot time and cups is less clear about authentication. But 11.10 is not LTS and not viable for long term deployment. My best hope is that some bugs will be fixed after release but based on past experience this is highly unlikely.

Computer Lab on the Cheap

Budget – a bit over $3000. Goal – a 26 seat computer lab for a high school. Is it possible?

Requirements

The computers need to run basic office productivity, web browsing software, authentication, and monitoring what students are doing.

Software

I went with Linux Terminal Services Project (LTSP). It doesn’t require a hard drive on the clients. The Ubuntu LTSP wiki is a good resource for setting this up. I tried using Proxmox (openvz) at first so I could have 2 LTSP nodes mixed on the same hardware as less intensive applications. This became problematic as openvz doesn’t easily support fuse. When I plugged in a USB drive to a thin client it wouldn’t work. I could have tried to build my own kernel with openvz and fuse support but decided against it. Instead I placed everything on one dedicated server.

For authentication I use existing OpenLDAP and samba servers. I won’t go into specific on setting these up in this post. I will say use the Ubuntu server guide and not the community wiki for openldap. The wiki is terribly out of date and in many cases incomplete or wrong. LTSP uses the underlying authentication so if you can log into your server with ldap you should be good to go. I didn’t find anything different about LTSP. pam_mount, pam_ldap, etc all work as they would on a traditional desktop. I used pam_mount to mount the student’s document folder automatically. I wrote about pam_mount before when messing with Active Directory.

For printing I used cups 1.5. Cups 1.4 in Ubuntu has some nasty bugs that make the server hang in some cases in my experience. I used an existing cups server but it could also just exist on the LTSP server. By using a central cups server though you can have non thin client desktops use it as well. This makes administration pretty easy. I will say I hate how it always show all printers even ones the user doesn’t have permission for! Also just set your MaxClients to some really high number! I found you can reach the default (100) pretty easily even with just 40 or so users! Why is the default so low? A modern computer can probably handle millions of clients.

To customize the user interface, just drop files in /etc/skel. Setup the desktop how you want it and copy in ~/.config

To view what students are doing you can use italc. The wiki makes it sound super easy to install on LTSP. The wiki rather confusing. I put the lts.conf file at /var/lib/tftpboot/ltsp/i386/lts.conf The man page on lts.conf doesn’t say anything about START_ITALC. My file looks like this:

[Default]
 START_ITALC = True

Windows Applications on LTSP

While I would have rather have used LibreOffice, it was determined we needed Office 2007. This is somewhat of a project of it’s own. For personal use wine works great with Office. For multiple, non-technical users? Forget it! I used Crossover Professional which comes with multi-user support and hooks to do extra scripting. Of course it does cost money and I sank about a third of my budget into this. Even with crossover be ready for lots of bugs and hacks. My issues included

  • Windows desktop is not the actual ~/Desktop folder! Why not!? I used a script to symlink them. I named by script according to their convention 01.create-stub so it gets called when a user logs into the first time and gets a new MS Office “stub”, which is just the user specific parts owned by the local user instead of root. In regular wine you would only have this and thus you can’t have multi-user support.
    #!/bin/sh
    rm -rf “$WINEPREFIX/dosdevices/c:/users/crossover/Desktop”
    ln -s -f “$HOME/Desktop” “$WINEPREFIX/dosdevices/c:/users/crossover/Desktop”
  • Can’t print to PDF. wine/crossover won’t allow it. It sort of works if you install cups-pdf but it’s too buggy for production use. Office 2007 has an add-on to save directly as PDF. So install it and any SP’s for Office.
  • wine remembers every printer forever! If you connect a printer via cups and remove it, Office will forever think it still exists. Again fine for personal use but in a school forget it! And what if you ever want to rename printers in cups? Duplicates. Delete all printer references in /opt/cxoffice/support/ms_office/system.reg and user.reg. At least this way the user starts out with no dead printers. I need to make a script to delete them on startup or something, it’s on my to do list.
  • Default applications – It doesn’t work. I wanted to make .doc open by default with Office. Crossover people tell me it’s a bug in Ubuntu. You can fix it by editing
    /etc/X11/Xsession.d/55gnome-session_gnomerc
    Change
    XDG_DATA_DIRS=/usr/share/gnome:/usr/local/share:/usr/share
    to
    XDG_DATA_DIRS=/usr/local/share:/usr/share/gnome:/usr/share
  • Lots of minor glitchy bugs such as the “Office button” menu often disappearing.
  • Excel uses a mdi form that doesn’t display correctly on the gnome task bar. I have no solution to this other than starting new Excel processes (by starting Excel each time, not just opening xls files). Why do MDI forms still exists? This is not DOS. We have operating system level window management these days.
Finally you can package your wine “bottle” as a deb for easy deployment. Fun fact – because Word is a single process application if you log into 2 LTSP as the same user and both run it, two windows will come up on the first users screen and none on the other! But with Excel you can run multiple processes. Ha and people say LibreOffice has a messy code base. This doesn’t happen when using different users so it’s not a problem.

Hardware

I find it pretty easy to come across donated Pentium 4 desktops and servers. My dedicated server was a HP ProLiant G5 with 16GB ram and two four-core Xeon processors. That’s 8 cores total. A very nice server. One could probably get away with using slightly older models or could just add many into a cluster. For a non profit it seems reasonable to me that you can find such equipment for free, though I do live in NYC which might make it easier as there are not shortage of mega financial firms with cycling through equipment.

Old desktops are gross, so hide them under the desk if possible.

Flat screen monitors are usually never given away. Be prepared to buy them. About a 1/3rd of my budget went to this and keyboards/mice.

It’s easier to find it easier to get old non networked printers. If they have a parallel port you can buy a “Print Server” and get it online.

Networking

I ran 2 gigabits of bandwidth from the server to the lab over two 1 gigabit cables/switches. Check out the Edubuntu LTSP requirements page. I used 2 switches in the lab each with a 1 gigabit backend and 100 megabits to each thin client. Another third of my budget went into networking equipment. Thin Clients are of course going to be network intensive! I found some volunteers to help with running cables and computers. Thanks volunteers!!

Expenses

The Volunteers and I

Results

LTSP runs fast with the right hardware! If you ever used VNC (please don’t it’s terribly slow!) you might be scared of remote access. LTSP uses X’s natural networking capabilities and it might not be as fast as RDP, but it’s close. Mouse input is very smooth. Programs start up super fast (they are probably all already in ram). I’ll probably try playing around with deploying some fat clients on some of the better machines to increase performance.

If you read my other posts you know I really get annoyed with all the Linux Desktop bugs. But most of that is mitigated by using thin clients. The server is always on and there are no laptop users. Syncing files and fighting upstart just becomes a moot point.

Installing new application is dead simple. I can apt-get install them on the ltsp server and they just appear even if a user is already logged in.

I just set up the lab, so I’ll update this post with results of actual student usage! And be sure to donate to Cristo Rey Brooklyn High School so we can get more computers! Otherwise I’ll assume you hate education.

Django End User Defined Custom Fields

I want a way for end users to add fields to my Django app without programming. I’ll publish this eventually, but here how to implement it. Latest version is now at http://code.google.com/p/django-custom-field/

First I made 2 models. One for the field and one for the values associated with the field.

from django.contrib.contenttypes.models import ContentType
from django.db import models

class CustomField(models.Model):
    """
    A field abstract -- it describe what the field is.  There are one of these
    for each custom field the user configures.
    """
    name = models.CharField(max_length=75)
    content_type = models.ForeignKey(ContentType)
    field_type = models.CharField(max_length=1, choices=(('t','Text'),('i','Integer'),('b','Boolean (checkbox)'),), default='t')

    def get_value_for_object(self,obj):
        return CustomFieldValue.objects.get_or_create(field=self,object_id=obj.id)[0]

    def __unicode__(self):
        return unicode(self.name)

    class Meta:
        unique_together = ('name','content_type')

class CustomFieldValue(models.Model):
    """
    A field instance -- contains the actual data.  There are many of these, for
    each value that corresponds to a CustomField for a given model.
    """
    field = models.ForeignKey(CustomField, related_name='instance')
    value = models.CharField(max_length=255,blank=True,null=True)
    object_id = models.PositiveIntegerField()
    #content_type = models.ForeignKey(ContentType)

    def __unicode__(self):
        return unicode(self.value)

Next I wanted this to work with admin. So I extend any ModelAdmin you want with this I need. Notice we don’t handle errors! I have my 3 simple data types all do client side validation, then if the server validation comes up invalid, it just throws them away. It’s less than ideal but I wasn’t sure how to get it to display the errors correctly.

from django import forms
from django.contrib.contenttypes.models import ContentType
from django.contrib import admin
from django.forms.widgets import TextInput

from models import *

class NumberInput(TextInput):
    input_type = 'number'

class CustomFieldAdmin(admin.ModelAdmin):
    def __create_custom_form(self, obj_id=None):
        custom_fields = CustomField.objects.filter(content_type=ContentType.objects.get_for_model(self.model))

        custom_form = forms.Form(prefix="cstm")
        for field in custom_fields:
            if field.field_type == 'i':
                custom_form.fields[field.name] = forms.IntegerField(label=field.name, required=False, widget=NumberInput(attrs={'style':'text-align:right;','step':1}))
            elif field.field_type == 'b':
                custom_form.fields[field.name] = forms.BooleanField(label=field.name, required=False)
            else:
                custom_form.fields[field.name] = forms.CharField(label=field.name, max_length=255, required=False)
            if obj_id:
                value = CustomFieldValue.objects.get_or_create(field=field,object_id=obj_id)[0]
                custom_form.fields[field.name].initial = value
        return custom_form

    def render_change_form(self, request, context, *args, **kwargs):
        context['custom_form'] = self.__create_custom_form(context['original'].id)
        return super(CustomFieldAdmin, self).render_change_form(request, context, *args, **kwargs)

    def save_model(self, request, obj, form, change):
        custom_form = self.__create_custom_form()
        custom_form.data = request.POST
        custom_form.is_bound = True
        if custom_form.is_valid():
            data = custom_form.cleaned_data
            for key,data_field in data.items():
                custom_field = CustomField.objects.get_or_create(content_type=ContentType.objects.get_for_model(self.model), name=key)[0]
                custom_value = CustomFieldValue.objects.get_or_create(field=custom_field,object_id=obj.id)[0]
                custom_value.value = data_field
                custom_value.save()
        # Hope that client side validation works since we don't handle errors here!

        return super(CustomFieldAdmin, self).save_model(request, obj, form, change)

Now we have to edit the admin template. This isn’t ideal but I couldn’t think of any other way. Edit change_form.html and add

{% include "admin/includes/custom_field_fieldset.html" with custom_form=custom_form %}

wherever you want. I say it’s not ideal because if you make lots of customizations it’s hard to keep track. Blocks help with this, BUT Django’s admin content block is pretty big, and I wanted to add mine basically in the middle of the content. Now you need the referenced custom_field_fieldset.html

{% spaceless %}
{% if custom_form.fields %}

  <fieldset class="module">
    <h2 class="collapse-handler">Custom Fields</h2>

    {% for field in custom_form %}
      <div class="row cells-1 {{ custom_form.prefix }}-{{ field.name }}">
        <div class="column span-4">
          {{ field.label_tag }}
        </div>
        <div class="column span-flexible">
          {{ field }}
        </div>
      </div>
    {% endfor %}
  </fieldset>

{% endif %}
{% endspaceless %}

That’s it. So now for the real test, can I add a custom field to my custom field model?

Now you can customize your fields while you customize fields! If you need to access the custom fields programmatically you can make shortcuts like

def get_custom_fields(self):
    return CustomField.objects.filter(content_type=ContentType.objects.get_for_model(Whatever))

Next I need to add integration with my applications import tool, make something so you can extend Model to get that helper function, and integrate with django-admin-export. Maybe even create a dropdown field_type which would require another model to store the data in. Then I promise to post to pypi.