Bookie Weekly Status Report Returns! – April 15 2012

Ok, I’m overdue for a ‘weekly’ status report. I’m going to try to kick this back into gear as it helps you out there track things and me feel like I’m moving forward by writing down all the little things I’ve done over the last bit.

Trello board to keep up to date: https://trello.com/board/bookie/4f18c1ac96c79ec27105f228

New Projects

In an effort to add some features to Bookie I’ve ended up starting two new repos of code meant to interact with Bookie.

  1. Bookie Parser

This is meant to start taking over the work of reading the page content and readable parsing the important content out. It was a chance to play with Tornado and Heroku. This also means that in the future I’ll be able to scale out the readable processing serperatly from the main Bookie website and host. It’s pretty bare bones right now and doesn’t directly talk to Bookie, but I’ll look at adding that integration soon as the API stabilizes and I get more tests going in it.

So far the Heroku bit has been pretty awesome. I have to deal with the fact that the app gets shut down and has to restart on first request, but hopefully that gets better as traffic and use picks up. You can tinker with it at http://readable.bmark.us

  1. Bookie Api

I’ve been wanting to start up a command line client for some of the Bookie work. The big thing is that I need tools to help manage invites and such. So it’s currently very admin centric, but eventually I’d like to get this into a ncurses cool command line interface to pull up recent bookmarks and even do some quick searches via the API. Aren’t API’s cool. This will also contain the reference Python API implementations so we’ll have two implementations soon. One in JS and one in Python.

I’ve got a beta version (which is really an alpha) up on PyPi so you can

$ pip install bookie_api
$ bookie ping

Build baby build

I spent some quality time with http://build.bmark.us to get the JS tests running via grover and phantomjs and that's awesome. I also added the new projects into the builder as well. So, while I don't have all the tests I need, at least now the ones I do have run consistantly.

Other little tweaks

  • Prettied up the new user invite email and landing page
  • Fixed a bug with dupe tags in the tagcontroller
  • Added more icons from the fontawesome set to pretty up the ui, especially the account page.
  • Lots of changes to the make/build steps for JS and CSS including actually doing the pyscss transition.
  • Everything is now on the final stable release of YUI 3.5. It's been a good ride through the development releases.

Upcoming events

I'll be giving a talk at Penguicon on using YUI for JS app development. If you're in the area stop by. This is Friday April 27th, at 6pm. Then on Saturday I've got a Bookie mini-sprint going on. I'll probably be hacking most of the weekend. Feel free to stop by and check things out.

A few ideas, quick ways to get on the Bookie contrib list

Quick ideas for improving Bookie

Ideas!

Well, with all the great stuff going on with Bookie, I’ve gotten a bit buried in some big changes. The background processing and importing updates are going to take a bit to get right.

This means, there’s a great chance for others to hack up the little tweaks that we need to really add some polish to Bookie. So below I’ve listed a few ideas that should be pretty simple things to add, but with a really good positive and visible effect on the site.

  • Add notification that user has invites

    Now that invites are there, we should highlight a user’s account navigation link to let them know they have invites available. I’ll periodically add them to the system, and we don’t want users to have to go to their account page each time to see they’ve got invites. I think a simple adding of one of the envelope/message type icons from our font-icon set would be perfect, with some sort of hover message to start. We might also want to highlight the block in their account page so it stands out that the invites are available.

  • Flash message system.

    We want to be able to let users know things have happened successfully after doing something that redirected them. Imports are going to be doing this, saving/updating bookmarks, etc. It’d be nice to have a consistent type of ui to drop flash messages in and them to show after a redirect.

  • Show new user message if self bookmarks page has no results

    When a new users starts up and logs in, they default to their own page of bookmarks…which is going to be empty. So we should detect this in our JS code that fetches the results and displays a set of default content with links to things like importing instructions, where to get the chrome extensions, and other handy new user tips.

    Some of this might also be nice to use for the email that a new user gets when they’ve been invited to Bookie.

  • Add firefox bookmark importer

    Ok, so this one is a bit more involved, but really, it’s a single class and a couple of Python methods. The hard part is reading in and figuring out how to match bookmarks to tags in Firefox’s JSON dump of bookmarks. Once we get the Firefox extension rolling, it’ll be great to have a good import system for the browser as well.

Well, here are four things I’d love to see happen in the near future to help make the experience a level nicer for everyone. If you’re interested in all or have any questions, ping me in #bookie in irc or shoot me a comment below. I’d be happy to help walk anyone through these or any other ideas you might have.

Bookie PyCon 2012 Sprint Report

http://www.flickr.com/photos/66176388@N00/4592858614

Sprint like mad!

So PyCon sprints, what can you say? You go, you hack…and hack…and some some point you take a break for a beverage…and hack some more.

Last year I got the real movement behind Bookie. So this really marks the one year anniversary for Bookie. I’ve had it as a side project to hack on in my spare time for the last whole year. In that time, honestly it’s not crazy different from where it started. However, it’s gone through multiple JS rewrites, two different UI designs, and a whole lot more. I’ve really learned a lot about development, testing, and making some hard choices over the last year. I hope by this time next year I’ll have announced Bookie on some big site (reddit/hacker news?) and survived.

This sprint though wasn’t the time. So what did I get done?

tl;dr

Bookie got improvements

  • Better JS tests
  • Better PY tests
  • Start of HTML5 history
  • Invite system
  • Threaded content fetcher
  • Start of celery background runner

First, I started out by working on getting the html5 history stuff going. It’s not perfect yet, but it’s started and I really realized I needed to have a better way to do JS tests, so…

Next I redid the JS tests. I don’t want to have to fire up the application in order to run my JS tests. I also don’t want to have to hit the database and such. This means I had to change the API tests. Rather than making real requests/responses, I test that the classes build the right type of requests. I verify the url, data payload, etc are correct.

Once the JS tests were redone, I realized that I hated how I had yXXX.js as the filenames and redid those as well. While I was cleaning up I dumped a bunch of old code we no longer needed. Basically tons of gardening cleaning out the weeds.

With that out of the way, the next day of sprints was all about getting an invite system underway. I originally wanted to do a throttled signup process, so anyone could sign up, but then I realized that really, invites will work better. The people using Bookie now will know who’d be interested in testing and if someone really wants to get in, they’ll contact me in some fashion.

With that up, I got to spend the next day fixing bugs in everything. Wheeee! What was cool was that I managed to get a few people at the sprints curious about Bookie and testing things out. Nothing exposes bugs like new users. During this process I spent some time cleaning house on the Python side of things and making tests easier to run/write.

Finally, I’ve started work on the background processing using Celery. I’ve got a big hurdle in that, but my cron’d stats processes are working and I’ve almost got imports running as out of process celery children. That should really help with new users. You know at some point they’ll come flooding in right? :-)

Overall, while I didn’t get a ton of new user facing features going, I did a TON of clean up and maintenance. As one person expressed “Wow, there’s a lot more in here than I expected when you said it was a bookmark application”. Bookie has really grown over the last year and she needs me to spend some time giving things some love before moving forward too fast. The sprints really gave me a chance to do that, all while hanging out and chatting with really smart people. What more can I ask for?

Bookie Sprint! Feb 25th 10am-4pm

What’s that I hear? A Bookie sprint is going to take place. We’ve got a clear mission, to get a Firefox extension going for Bookie. Since all of the rework has been going into the JS driving Bookie, the hope is that we can get a Firefox extension to do everything the Chrome one can do.

We’ll be playing with the newish Firefox Add-On SDK to build it and it should serve as a great run down of the new JS code for everyone involved.

The new project for the Firefox extension is starting up at: https://github.com/mitechie/bookie-firefox

When is this fantastic event you ask? Well 10am-4pm at my place. If you’re not sure where that is, just ask me. You can also participate virtually in #bookie on Freenode.

Firefox extensions not really your thing? That’s ok, I’m sure there will be lots of side hacking going on and with everyone in the same room, we should be able to crank out some great work. Check out the Trello board for ideas on things that need to happen before the Bookie 0.4 release is out. Mobile-izing the view, updating docs, adding features to the Chrome extension, are just a few of the fun things to hack on.

Book status report…the JS UI is alive!

Well that took a while. Back in September I opened a branch of Bookie to try to do some cool Backbone driven UI stuff. I decided that maintaining a separate mobile UI was going to kill me. So I needed something I could tweak to make mobile friendly without dual sites. Between a JS drive UI and responsive design techniques for the CSS, I should be able to make things not suck so much.

So today, the first part of that has gone live. https://bmark.us now has a UI driven by API calls through Javascript. It’s using the YUI MVC stuff that’s in their 3.5pr. You might have noticed that Backbone isn’t in there any more. I started the app with jQuery thinking it would be a bit more friendly to outside contributions. However, after trying to put together jQuery, Backbone, HistoryJS, etc. I kind of got tired of making due and moved to YUI. It’s my choice for JS frameworks and since there’s not exactly a pouring of external commits, moving to YUI doesn’t hurt me much.

So finally, after waaaay to many months I feel like the site is moving forward. An updated UI that needs some responsive love. An API that need some more methods, logging, etc. Phew.

There are a number of bugs and tweaks that still need working on. I’m going through them and to help be transparent I’ve started a public Trello board for Bookie so that everyone can see exactly what I’m in the process of, what’s next on the board, etc. Hey, if you see something on the board that’s on your hot button list, feel free to take it and run with it.
Patches welcome :-)

CoffeeHouseCoders 11/23/11: YUI Theater group viewing

Just a heads up, this week’s CoffeeHouseCoders (CHC) Detroit-ish will be a bit different. One of the goals of moving the location to the new Caribou was that we get access to the meeting room. This opens up the opportunity for us to have some group discussion and such around various topics. We’re going to give that a shot this week with a group viewing of YUI Theater video viewings and JavaScript discussion.

Most of us do at least some JavaScript in our work and projects so I think it’s relevant and should be fun to geek out before the holidays start up. I’ll have a little projector and speaker and with the new videos from YUIConf 2011 going up, it’ll be nice to set aside some time to catch up on some of the recorded presentations. Take a peek and set aside one or two "must watch" videos for Wed night! Not all of the videos are YUI specific, so it should be useful for all of us doing JavaScript.

Launchpad Team: Day 1 complete

Phew, well one day down. I dove head first into Canonical and Launchpad today. It’s a bit amazing the amount of information and parts there are to everything. Everyone welcoming me throughout the day was great, but my head is still spinning a bit for sure.

I managed to get a nice starter walk-through of Launchpad and find my way through a superficial bugfix and merge request. So hey, that wasn’t so bad heh. It’s kind of exciting to throw out all my usual tools I’ve been mastering for a while and start over. Make files, zpt files, ZCA, and YUI run the show. Time to see how people get things done without Fabric, Mako, and SqlAlchemy.

I’m really excited to get to some real change and hope to pick things up quickly. I know a while ago I was disappointed that Launchpad wasn’t taking advantage of some of the Javascript driven UI enhancements that we can do these days. The change of that is already in full swing and my team is looking to land a nice chunk on the bugs UI shortly. Let’s get to work!

Bookie status report: Aug 14th 2011

Updates this week

Finally, the new and fancy Bookie API is reworked and better than ever. The live running site is updated to use the new api and there’s a new chrome extension that’s update for it as well. Make sure you get the latest dev version of the Chrome extension.

With the api updated, it’s time to get to work on finishing up 0.3 for release. This means that we need to finish updating the mobile site and some rework of how the readable content is stored. Since we pull the content out of the page based on actual page you see, we should store these separated from user to user. We don’t want to leak logged in info from one user to another.

Farewell Firefox

With the api update, the Firefox is so far out of date as to be unusable. There’s some work going on to help catch it up, but I think I’m going to remove it as a feature in the docs and such. We just can’t claim it at this time.

New testing

With the new api we hit some testing issues. Since there’s no good way I’ve found to functional test the extension, the tests aren’t great and reliable. I also don’t have tests setup on some shared code like the bookie.api.js code. I’m thinking of setting up a running instance with some live data with the sole purpose of running tests on the JS code against it. We might even go so far as to setup the extension html files as an actual website and write some functional tests around that. After all, I can load popup.html and do some testing of that as a normal web page. We’ll be missing the interactions with background.html and such, but might be worth some attempt.

Any ideas and help with this is most welcome.

Alpha Testing

We have a signup for if you’re interested in alpha testing the hosted install at https://bmark.us. If you’d like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

  • Current Chrome Extension: 0.3.11
  • Most recent code updates: develop branch
  • Current working branch: develop

YUI, how dare you make me rethink

So I’ve been spending time working on moving ahead with my project using YUI. So it’s been a lot of trying to port over old jQuery code into YUI, going through docs on the various widgets and plugins, and just trying to figure out how to rearrange how I think with this stuff.

The hard part is that YUI3 enforces something we all consider good practice. Don’t pollute the global namespace. What’s the bad in that? Well, it presents some challenges with the way I used to do things.

MyApp = {};

MyApp.home_page = {

    'init': function () {
        // do some stuff here
        // like load some social content or crap
        MyApp.home_page.bind_stuff();
    },

    'bind_stuff': function () {
        // hey, we have fancy events we bind/catch
        // woooo
    }
};

// and then in the page onReady I just
MyApp.home_page.init();

So that's how I have been doing things. Parts get abstracted out, shared, but basically each page tends to have an object for each page and any extra handling of events or whatever that requires.

However, YUI3 does this cool thing that breaks that. When you use YUI3 you have a YUI().use() and everything gets encapsulated within that bit of code. So if I were to just move the MyApp code into the YUI block I'm no longer able to access it from the page of content I want to run JS on. So this is good, no JS globals, but it's bad...not the way I've been doing things.

So I've had to figure out how to adjust my old code to the new YUI3 way, and I came across this great video on structuring your code a bit better. It makes some sense, but it seemed to just scratch the surface. Unfortunately, his example is a single page application so it's a bit simpler than what I needed, but it got the point across for me.

So what's the new stuff look like? It looks like YUI3 modules

YUI.add('myapp', function (Y) {
    var MyApp,

    MyApp = function (config) {
        MyApp.superclass.constructor.apply(this, arguments);
    };

    /** Static **/
    MyApp.NAME = 'MyApp';
    MyApp.ATTRS = {

    };

    Y.extend(MyApp, Y.Base, {
        initializer : function (config) { },

        destructor : function () {},

        /** Public Methods **/
        'some_page': function () {
            var that = {};

            that.init = function () {
                // do some stuff
                that.bind_stuff();

            };

            that.bind_stuff = function () {
                // and we're doing stuff
            };

            // return the page object with all the controls/method
            return that;

        } 

    });

    Y.namespace('MyApp');
    Y.MyApp.Window = MyApp;

}, '0.1', { requires: ['base', 'node', 'my-ui', 'yui2-datatable', 'yui2-button'] });

// and then I use it from my page with
YUI({ }).use('myapp', function (Y) {
    win = new Y.MyApp.Window();
    my_page = win.qcontent_admin()
    my_page.init();
});

So this has me creating a module for my application code. It allows me to specify versions, requirements, etc. It keeps the YUI3 way of not polluting any global namespace for the application, and while a bit more verbose, seems to be working out. I like this since it's getting to what I was talking about back before when I was looking at moving to YUI from jQuery. This feels more like treating the JS code like application code.

I could split up the code into different modules based on different logical sections of the application and I might be able to reuse bits of this down the road. For instance, we use a common base bit of application code for our authentication across apps. I could break out the JS code for the auth pages
into their own module, version it, and reuse it much easier.

What do you think? Am I heading in the right direction or am I just making life more complicated than it's supposed to be?

Howdy YUI, looking fine…ish

I subscribe to some RSS feeds and I’ve seen some cool videos/posts about YUI3 and it looked kind of interesting. I’ve always felt like I should be using/checking out YUI more. Everything seems very well thought out, tested, and I love that they have a whole package. Everything from DOM tools, Ajax, CSS Grids all the way to ui widgets. I’ve always thought they had the best autocomplete widget around from the demo. Tab should complete the current selection, it just should.

So I picked up a copy of YUI2.8 Learning the Library and while it’s not YUI3, it’s a good view into the minds of things and how they fit together. So I decided to port over a work project I’m working on to YUI. It’s kind of CMS-y so I want to setup grids just to help flexibility as things go on. (Although…I’m a grid hater for the record) I also want to take advantage of the UI collection for auto complete, progress bar, text editor, treeview and possibly datagrid.

Today was day one and so far so good. It takes a bit of work to get your head moved over from jQuery. I’ve been using it for a while and really the thought process in jQuery is different. Go find about any tutorial and you’ll see things like:

// find all the instance of class="important"
// and then make sure they're marked with a bolder font
$('.important').css('font-weight', 'bold');

This view is kind of against the ideas of mixing logic and template, in a way. I
mean this is all view information here. Instead I think I should have things
more like:

    var highlight_important = function (identifier) {
        var default_id, id;

        default_id = '.important';

        if (identifier !== undefined) {
            id = identifier;
        } else {
            id = default_id;
        }

        $(id).css('font-weight', 'bold');
    };

Now, ideally this would take more config options for just what class property to set and what value to set, all optional of course. And you have this reusable component with a sensible function name you can reuse throughout the app.

This is probably a bad example, but the point is that jQuery comes across as 'find the html and do stuff' where frameworks that come across more verbose/complex, like Dojo and YUI, seem to present things as 'write JS logic' instead.

For instance, I love the idea of the modules in YUI3. I used to just have a custom namespace object like:

mystuff = {
    
    'ui': {
        'zebra': function () {
            ...
        },
    },

    'form': {
        'init': function () {
            ...
        },
        
    }

This is fine, but I like the idea of the modules you can treat like other YUI modules. For instance my form code turns into something like:

YUI.add('myform', function (Y) {

    Y.myform = function () {
        // private property to track the forms we've processed
        forms_formatted = {};

        var that = {};

        that.init = function () {
            Y.myform.maxlabels();
        },

        that.dosomething = function () {
            ...
        }

        return that;
    }();
}, '0.0.1', { requires: ['node'] });

// and use it like
var login_page = function () {
    YUI().use("node", "yui2-button", 'myform', function(Y) {

        var Y2 = Y.YUI2;

        //YUI 2 implementation code
        var button = new Y2.widget.Button("submit");

        // init the form ui tweaks
        Y.myform.init();
    });
};

This also shows off how I can do things like include the YUI2 widgets.

So people are asking "Why use YUI over jQuery" and it really comes down to 'total package' with the overall completeness of things, 'well thought outness' of the library. In reading things I'm always hitting points like "man, that's a really good way to do/handle that". Finally, I want to try out and seem to like how it makes you 'think' when writing the code.

Will it stick, who knows, but I've learned enough JS to want to try out something a bit higher up. I consider it a bit like going from webob to Pylons. They're both JS underneath and all, but one includes a lot more bits than the other at the cost of some framework complexity.