Tech Rantings from a Michigan Techie

Book status report…the JS UI is alive!

mitechie — Sun, 22 Jan 2012 19:27:51 +0000

Well that took a while. Back in September I opened a branch of Bookie to try to do some cool Backbone driven UI stuff. I decided that maintaining a separate mobile UI was going to kill me. So I needed something I could tweak to make mobile friendly without dual sites. Between a JS drive UI and responsive design techniques for the CSS, I should be able to make things not suck so much.

So today, the first part of that has gone live. https://bmark.us now has a UI driven by API calls through Javascript. It’s using the YUI MVC stuff that’s in their 3.5pr. You might have noticed that Backbone isn’t in there any more. I started the app with jQuery thinking it would be a bit more friendly to outside contributions. However, after trying to put together jQuery, Backbone, HistoryJS, etc. I kind of got tired of making due and moved to YUI. It’s my choice for JS frameworks and since there’s not exactly a pouring of external commits, moving to YUI doesn’t hurt me much.

So finally, after waaaay to many months I feel like the site is moving forward. An updated UI that needs some responsive love. An API that need some more methods, logging, etc. Phew.

There are a number of bugs and tweaks that still need working on. I’m going through them and to help be transparent I’ve started a public Trello board for Bookie so that everyone can see exactly what I’m in the process of, what’s next on the board, etc. Hey, if you see something on the board that’s on your hot button list, feel free to take it and run with it.
Patches welcome

Ubuntu Community Appreciation Day: My Loco!

mitechie — Sun, 20 Nov 2011 20:53:42 +0000

So on Ubuntu Community Appreciation Day I want to toss a big thanks out to the Michigan Loco. It’s a great bunch of guys and gals that I talk with online every day and have helped keep me sane, taught me new things, and overall have just made this community thing work for me. If it wasn’t for them, I’d not be running Ubuntu and working on Launchpad today. So hats off to everyone in the Loco and here’s to all the other great people making this community rock!

An updated email config, 2 offlineimap, mutt, and dovecot ftw!

mitechie — Sun, 20 Nov 2011 20:48:37 +0000

Since joining the Launchpad team my email has been flooded. I’ve always been pretty careful to keep my email clean and I’ve been a bit overwhelmed with all the new mailing lists. There are a bunch of people working on things, as you can imagine. So the email never stops. I’m still working on figuring out what I need to know, what I can ignore, and what should be filed away for later.

Another thing I’m finding is that I’ve got emails in both of my accounts around a single topic. For instance, I have to do some traveling. I’ve got emails on both my Gmail (personal) and Canonical (work) accounts that I really want to keep together in a single travel bucket.

I currently have offlineimap pull both of work and personal accounts down into a single folder on my machine ~/.email/. So I’ve got a ~/.email/work and a ~/.email/personal. I use mutt then to open the root there and to work through email. It works pretty well. Since I really wanted a global "travel" folder, I figured I’d just created one. So that works. I end up with a directory structure like:

personal
travel
work

The problem

Of course the issue here is that when offlineimap runs again it sees the email is no longer in the personal or work accounts and removes them from the server. And the travel folder isn’t a part of any server side account so it’s not backed up or synced anywhere. This means Gmail no longer sees things, my phone no longer sees them, and I’ve got no backups. Oops!

Solution start

So to fix that, my new directory structure needs to become an account. So I setup dovecot on my colo server. This way I could have an imap account that I could do whatever with. To get my email into there, I setup offlineimap on my colo to pull personal and work down as I had on my laptop. So I still have things in a ~/.email that’s from the accounts and then dovecot is keeping all of my email in ~/email (not a hidden dir). To get my email into there, I symlinked the ~/.email/personal/INBOX to ~/email/personal and did the same with the work account. Now the two accounts are just extra folders in my dovecot setup.

So there we go, colo is pulling my email, and I changed my laptop to offlineimap sync with the new dovecot server. In this way, I’ve got a single combined email account on my laptop using mutt. I then also setup my phone with an imap client to talk directly to the dovecot server. Sweet, this is getting closer to what I really want.

Issues start, who am I

Of course, once this started working I realized I had to find a way to make sure I sent email as the right person. I’d previously just told mutt if I was in the personal account to use that address and if in the work account use that one. Fortunately, we can help make mutt a bit more intelligent about things.

First, we want to have mutt check the To/CC headers to determine who this email was to, if it was me, then use that address as a From during replies.

mutt config:

# I have to set these defaults because when you first startup mutt
# it's not running folder hooks. It just starts in a folder
set from="rharding@mitechie.com"
# Reply with the address used in the TO/CC header
set reverse_name=yes
alternates "rick.stuff@canonical.com|deuce868@gmail.com"

This is a start, but it fails when sending new email. It’s not sure who I should be still. So I want a way to manually switch who the active From use is. These macros give me the ability to swap using the keybindings Alt-1 and Alt-2.

mutt config:

macro index,pager \e1 ":set from=rharding@mitechie.com\n:set status_format=\"-%r-rharding@mitechie.com: %f [Msgs:%?M?%M/?%m%?n? New:%n?%?o? Old:%o?%?d? Del:%d?%?F? Flag:%F?%?t? Tag:%t?%?p? Post:%p?%?b? Inc:%b?%?l? %l?]---(%s/%S)-%>-(%P)---\"\n" "Switch to rharding@mitechie.com"
macro index,pager \e2 ":set from=rick.stuff@canonical.com\n:set status_format=\"-%r-rick.stuff@canonical.com: %f [Msgs:%?M?%M/?%m%?n? New:%n?%?o? Old:%o?%?d? Del:%d?%?F? Flag:%F?%?t? Tag:%t?%?p? Post:%p?%?b? Inc:%b?%?l? %l?]---(%s/%S)-%>-(%P)---\"\n" "Switch to rick.stuff@canonical.com"

That’s kind of cool, and it shows in the top of my window who I am set to. Hmm, even that fails if I’ve started an email and want to switch who I am on the fly. There is a way to change that though, so another macro to the rescue, this time for the compose ui in mutt.

mutt config:

macro compose \e1 "f ^U Rick Harding \n"
macro compose \e2 "f ^U Rick Harding \n"

There, now even if I’m in the middle of creating an email I can switch who it’s sent as. It’s not perfect, and I know I’ll screw up at some point, but hopefully this is close enough.

Firming up with folder hooks

Finally, if I know the folder I’m in is ONLY for one account or the other, I can use folder hooks to fix that up for me.

mutt config:

folder-hook +personal.* set from="rharding@mitechie.com"
folder-hook +personal.* set signature=$HOME/.mutt/signature-mitechie
folder-hook +personal.* set query_command='"goobook query \'%s\'"'

So there, if I’m in my personal account, set the from, the signature, and change mutt to complete my addresses from goobook instead of the ldap completion I use for work addresses.

Not all roses

There are still a few issues. I lose webmail. After all, mail goes into my Gmail Inbox and then from there into various folders of my dovecot server. Honestly though, I don’t think this will be an issue. I tend to use my phone more and more for email management so as long as that works, I can get at things.

I also lose Gmail search for a large portion of my email. Again, it’s not killer. On my laptop I’ve been using notmuch (Xapian backed) for fulltext search and it’s been doing a pretty good job for me. However, I can’t run that on my phone. So searching for mail on there is going to get harder. Hopefully having a decent folder structure will help though.

I’ve also noticed that the K-9 mail client is a bit flaky with syncing changes up on things. Gmail, mutt, and I’ve also setup Thunderbird all seem to sync up ok without issue, so I think this is K-9 specific.

That brings up the issue of creating new folders. Offlineimap won’t pick up new folders I create from within mutt. It won’t push those up as new imap folders for some reason. I have to first create them using thunderbird, which sets up the folder server side for me. Then everything works ok. It’s a PITA, but hopefully I can find a better way to do this. Maybe even a Python script to hook into a mutt macro or something.

Wrap Up

So there we are. Next up is to setup imapfilter to help me pre-filter the email as it comes in. Now that all email is in one place that should be nice and easy. I can run that on my colo server and it’ll be quick.

This is obviously more trouble than most people want to go through to setup email, but hey, maybe someone will find this interesting or have some of their own ideas to share.

CoffeeHouseCoders 11/23/11: YUI Theater group viewing

mitechie — Sun, 20 Nov 2011 19:26:43 +0000

Just a heads up, this week’s CoffeeHouseCoders (CHC) Detroit-ish will be a bit different. One of the goals of moving the location to the new Caribou was that we get access to the meeting room. This opens up the opportunity for us to have some group discussion and such around various topics. We’re going to give that a shot this week with a group viewing of YUI Theater video viewings and JavaScript discussion.

Most of us do at least some JavaScript in our work and projects so I think it’s relevant and should be fun to geek out before the holidays start up. I’ll have a little projector and speaker and with the new videos from YUIConf 2011 going up, it’ll be nice to set aside some time to catch up on some of the recorded presentations. Take a peek and set aside one or two "must watch" videos for Wed night! Not all of the videos are YUI specific, so it should be useful for all of us doing JavaScript.

Launchpad Team: Day 1 complete

mitechie — Tue, 15 Nov 2011 01:09:21 +0000

Phew, well one day down. I dove head first into Canonical and Launchpad today. It’s a bit amazing the amount of information and parts there are to everything. Everyone welcoming me throughout the day was great, but my head is still spinning a bit for sure.

I managed to get a nice starter walk-through of Launchpad and find my way through a superficial bugfix and merge request. So hey, that wasn’t so bad heh. It’s kind of exciting to throw out all my usual tools I’ve been mastering for a while and start over. Make files, zpt files, ZCA, and YUI run the show. Time to see how people get things done without Fabric, Mako, and SqlAlchemy.

I’m really excited to get to some real change and hope to pick things up quickly. I know a while ago I was disappointed that Launchpad wasn’t taking advantage of some of the Javascript driven UI enhancements that we can do these days. The change of that is already in full swing and my team is looking to land a nice chunk on the bugs UI shortly. Let’s get to work!

Mutli-monitor on the go, DisplayLink usb monitor

mitechie — Tue, 08 Nov 2011 02:56:51 +0000

Now that I’m going to be working for Canonical I’ve got to get ready for some week long sprints. Now, I LOVE my 12″ X201 thinkpad. It’s the best laptop I’ve ever owned, and I’ve owned macs, toshibas, dells, and larger thinkpads. When I’m home, I dock and use dual 21″ displays. When I travel, it’s the perfect size, yet still packs an i5 and 8GB of ram.

However, a week of living inside a 12″ display has me a little claustrophobic. So I decided I should do something about it. They make some decent looking USB powered external monitors that seemed like they’d travel pretty nice. Once cable, no external power needs, and another 1024px is a good thing. Mobile mutli-head..sweet!

Getting it working

The trouble is getting it working. The USB monitors use something called DisplayLink to work. Getting that to work is a little tricky. Fortunately, a few brave souls paved my way. You can see the links I checked out to get started

Once you install xserver-xorg-video-displaylink you’ll need an xorg.conf file since DisplayLink doesn’t work with all the hotplug business that makes modern video work.

It’s alive!

The problems

The real trouble is that I ONLY want this xorg.conf file when I’m actually using the USB display. When I’m docked at home or using a projector, I don’t need it. Right now I’ve just written a toggle script that I use to flip the xorg back and forth.

There are also some usage issues. It only works for me if the DisplayLink monitor is the primary one in the xorg server setup. This means the LightDM is running on there and it’s running in just a few hundred pixels of the display. I can still log in though so it’s not killer.

You can’t drag windows back and forth among them, which isn’t the end of the world, but some apps (like Google Chrome) only launch on the same display as the currently running instances. So I can’t find a way to get Chrome to run on both monitors. For now I just run Firefox on one and Chrome on the other.

Overall, I’m really digging the setup. I’m sitting at the bar with irc on a second display while I write this blog post out all unplugged from any power. I think it’ll make life a LOT nicer for road traveling for any extended time.

#!/bin/sh -e
# toggle xorg.conf on/off so I can add/remove it as needed
# requires a reboot after running to take effect

if [ -e /etc/X11/xorg.conf ]
then
    echo "Removing xorg.conf file"
    sudo mv /etc/X11/xorg.conf /etc/X11/xorg.conf.usb
else
    echo "Setting xorg.conf file"
    sudo mv /etc/X11/xorg.conf.usb /etc/X11/xorg.conf
fi

My xorg.conf file I’m using

############ Original Video Settings ###########

Section "Files"
    ModulePath      "/usr/lib/xorg/modules"
    ModulePath      "/usr/lib/xorg/modules/drivers"
EndSection

##################################################

Section "Device"
    Identifier      "Configured Video Device"
    Driver          "intel"
EndSection

Section "Monitor"
        Identifier      "Configured Monitor"
EndSection

Section "Screen"
    Identifier      "Default Screen"
    Monitor         "Configured Monitor"
    Device          "Configured Video Device"
    SubSection "Display"
            Depth   24
            Modes   "1280x800"
    EndSubSection
EndSection

############### DisplayLink Stuff ###############

Section "Device"
    Identifier      "DisplayLinkDevice"
    driver          "displaylink"
    Option  "fbdev" "/dev/fb0"
EndSection

Section "Monitor"
    Identifier      "DisplayLinkMonitor"
EndSection

Section "Screen"
    Identifier      "DisplayLinkScreen"
    Device          "DisplayLinkDevice"
    Monitor         "DisplayLinkMonitor"
        SubSection "Display"
            Depth   16
            Modes   "1024x600"
        EndSubSection
EndSection

#################################################

Section "ServerLayout"
        Identifier      "Server Layout"
        Screen  0       "DisplayLinkScreen" 0 0
        Screen  1       "Default Screen" RightOf "DisplayLinkScreen"
        Option          "Xinerama" "Off"
EndSection

Visual of the display issue with LightDM

Been a good summer, fitness, woodworking, and new job coming soon…

mitechie — Thu, 20 Oct 2011 00:23:19 +0000

I’m very late on a bunch of weekly status reports, but I’ve got good reason I promise. This summer has been a bit of a “summer of change” for me. Perhaps more renovation than change.

First, I decided I was sick of looking and feeling like crap and got into a weight loss program. I’ve finished that program today with some really good results. I’m down 37lbs, with another 19 to go until I hit my goal. Here’s hoping I can get there before the year is out. This has really helped me feel a ton better. I did a lot of biking this summer, including a 48mile monster that beat me up, but man was that fun.

Next up, I finally managed to clean the garage out. That means the woodworking tools are open for business. I’ve only started working on putting some finish on some new oak closet doors, but that’s a huge move forward. I’ve not done any woodworking since the boy was born and now that the space is cleared out I’m looking forward to getting back to making some shavings.

Finally, the big news. I’ve put in my two weeks notice with Morpace and have accepted an offer to work for the Launchpad team with Canonical.

I can’t express how excited I am to get this chance to work with a team of smart people that are really working hard to build something awesome. In many ways this is the culmination of a goal I set years ago to get paid to work on open source software, and not only that, but to get paid to work with Python as well.

Morpace was really good to me and I feel sad to leave them. They gave me my first chance to prove that I could be a productive Python developer. I’ve grown though and the opportunity to work with all of the great people at Canonical is just too much to pass up. I can’t wait to jump in and see what I can help with.

It’s been a good summer. I can only hope that each year continues to be as fulfilling.

Bookie weekly status report: Sept 18 2011

mitechie — Sun, 18 Sep 2011 17:20:04 +0000

Updates this week

Slow week. The changes to the fulltext search involved a small bug in updating the tag_str property of the bookmark. I got this fixed and added a migration to rebuilt the tag_str for all bookmarks in the system. The big sign of this was that the edit ui wasn’t showing the tags on the bookmark for edit.

Part of fixing that was a full update to SqlAlchemy 0.7 and moving all post/pre hooks into the new events system SqlAlchemy provides.

Along with that I also updated things to use the new release of Pyramid 1.2. This big thing with this is the new debug toolbar is available for development. It’s definitely pretty cool and helps provide some timing/insight as testing requests.

What’s next?

There’s a ton to work on. The celery task runner, the backbone.js in place
edit ui, and investigating using tagger to provide some tag
suggestions according to the content.

Alpha Testing

We have a signup for if you’re interested in alpha testing the hosted install at https://bmark.us. If you’d like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

Current Chrome Extension: 0.3.16
Most recent code updates: develop branch
Current working branch: develop

Bookie weekly status report: Sept 7 2011

mitechie — Thu, 08 Sep 2011 01:10:58 +0000

Updates this week

Sorry, I’m late on the weekly report. The holiday weekend threw me off. However, I’ve got good stuff for you. I completed the port of the fulltext search from being database specific to being Whoosh which is pure Python. This should be cool because it reduces code complexity trying to change things up based on the database used. It also been performing really well. The hosted http://bmark.us site is using it for everyone’s content.

Be aware, if you do the migration, it takes a while to run. It has to go through and processes every readable record one by one. So when you upgrade, let it run.

What’s next

Aside from that, I started playing with an all Javascript front end using Backbone.js. The idea is to see about setting up a very interactive front end that would allow in place edits of bookmark data easily using the existing API. We’ll see how it works out.

Alpha Testing

We have a signup for if you’re interested in alpha testing the hosted install at https://bmark.us. If you’d like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

Current Chrome Extension: 0.3.16
Most recent code updates: develop branch
Current working branch: develop

Bookie status report: Aug 28th 2011

mitechie — Mon, 29 Aug 2011 00:28:54 +0000

Updates this week

With 0.3 out and gone, it’s time to move forward on 0.4 tickets. I started out with the big one, edit/add form ui for bookmarks on the page. This now works and when you’re browsing urls from your page (/{username}/recent) you’ll see a new “E” icon next to the “R” one. You’ll also see a “Add” button at the top left.

Once we had the add/edit form, I could setup a bookmarklet to store your current page there. If you log into the site and go to your account page, a bookmarklet url is there for the taking. Drag it to your browser bookmark bar. On some platforms (hey Android!) you can’t save a bookmarklet to the bookmarks. You first have to create a normal bookmark, and then edit the link after the fact. To help that out, there’s a copy/pastable version of the bookmarklet on the account page. Just long press the text area, select all, and then paste into your bookmark from earlier.

It’s my hope that this bookmarklet will help users on Firefox and other browsers use Bookie more. It’s not the full extension experience, but hopefully holds people over for now. Note that when storing bookmarks in this way, the content is not immediately available as from the Chrome extension. The cron job that does fulltext parsing runs at 4am each morning.

Next I started some css work. The readable page no longer uses the Cabin font. It’s also got some default styles for H1 and such. Let me know if any content looks funny and we can look to add some additional styles going forward.

Next up

For the week ahead, I’ll be mainly working on tests. I don’t have enough for the new add/edit feature. I’m still working on adding live api tests to the phantom.js test suite. If I get time, I also hope to start work on a private bookmark feature.

Alpha Testing

We have a signup for if you’re interested in alpha testing the hosted install at https://bmark.us. If you’d like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

Current Chrome Extension: 0.3.16
Most recent code updates: develop branch
Current working branch: develop