The AppRabbit Blog

A place to talk about apprabbit.com and other cool technology related stuff.

New summary reporting functionality now available in AppRabbit!

I'm excited to announce that we've officially released the new Summary functionality in AppRabbit, so it's now easier then ever to setup views of your AppRabbit data to make it as useful as possible.  You can group by fields, aggregate related data, and filter on nested relationships, all from the new Summary View screen.  This functionality is available to admins by default and can be granted to specific groups of users on a per-model basis.  You can also create summary views and share them with other users, just like with regular saved views.  

In the comings weeks we'll be posting some tutorial videos for getting started with AppRabbit, and we'll be sure to include a video on setting up summary views of your data. 

How to fix FastCGI timeouts for long running scripts in Apache

I have been dealing with long running tasks timing out for quite a while now and I wanted to share the solution I finally tracked down that worked for me.  AppRabbit runs on Ubuntu Linux, with Nginx handling static content and acting as a reverse proxy and Apache as the app server, using mod_fcgi to run the actually Python/Django code.  The problem first arose during data imports, because they can sometimes take a while to run if users are importing a lot of data, but I resolved that issue by moving the processing to a back end process using RabbitMQ and Django-Celery.  But alas the timeout issue reared its ugly head again, this time when exporting large amounts of data, because it just took a while to prepare the data for exporting.  

The error messages that showed up in the logs were "mod_fcgid: read data timeout in 40 seconds" and "Premature end of script headers", neither of which are very helpful, but at least we know we're dealing with an issue of mod_fcgi timing out because the script is taking too long.  The solution is to edit your fcgid.conf file (which for me is located at /etc/apache2/mods-enabled/fcgid.conf) and add this line: 

FcgidIOTimeout 300

Once you've finished editing fcgid.conf and restart apache, this will allow your scripts to run for up to 300 seconds (5 minutes), and lets face it, at that point maybe it's time to refactor something.

How to keep the blank first option for required drop down lists in Django

When you define a Django field with Choices, it creates a drown list for you, and if you set that field as "required=True" then it will remove the blank option from the list and make whatever the first option is selected by default.  But people are generally lazy, and if a field already has a value selected they are just as likely to skip over it without actually setting it to the appropriate option as they are to take a moment and click on the drop down list to see what options there are.  So there are times when you might want a drop down list field to be required but still have the blank first option, so that the user is forced to actually click it and selected a choice.  To make this happen, simply slap this code in wherever you're instantiating your form object in Django:

form.fields['my_required_field'].choices.insert(0, ('', '---------'))

This will add a blank option to the drop down list for the field "my_required_field", while stil making it a required field.

How to beat the competition: Buy them a copy of Minecraft

At a local nerd get together the topic of Minecraft was a common discussion point, due in large part to the highly addictive nature of the game.  A quick Twitter search would indicate that this is the new World of Warcraft, in terms of addictiveness and customer loyalty.  I've noticed that this game really seems to attracts the problem solvers, the people who love a good challenging puzzle and will work at until they figure it out.  It's like a giant Choose Your Own Adventure puzzle, zombies and giant spiders included.

An interesting aspect of the game is that it is currently in alpha stage, so things are changing all the time, stuff that worked yesterday (like using ladders to climb down into water without drowning) may not work tomorrow if the developer "fixes" a glitch, while other things that don't yet work but hint at coolness (like fishing poles that don't do anything...yet) might provide some cool new opportunity tomorrow when the developer adds that functionality.  

Even when things aren't changing, there are such a huge number of possible combinations of events and materials to be discovered, it's truly awe inspiring.  Throw in a little bit of "my friend built a glass tunnel under the lake, I have to have my own glass tunnel under the lake!" and people building insane stuff inside the game like a 16 bit computer, and you've got a game that will really bring out the nerds.

But this wasn't meant to be a game review, or a sales pitch for Minecraft (or Minecraft Addicts Anonymous, as the case may be), in fact it's meant to be a warning to the storm weary sailors adrift on the sea of Startups: Don't waste all of your valuable time playing Minecraft while your competition is hard at work building something better than you.  I'm not saying don't take time for yourself, and I'm not saying Minecraft is bad, I'm just saying that if you spend your late night hacker sessions hacking at zombies in Minecraft instead of knocking out those features or fixing bugs, you're probably going to have a hard time keeping up with your competition.  

Which brings me to the main point of this blog post, which is to say that if you're working on a budding startup and you really want to put Minecraft to good use, buy a copy of it for each of the competition's developers.  It's like a tar baby for tech nerds.  In a weeks time all work will have stopped over at the competition and you can plug ahead with churning out new functionality and completely overrun the other guys.  Shoot, you may bring things to such a grinding halt that you could still put in an hour or two here or there playing Minecraft yourself, just as long as you remember that you're the pusher, not the addict.  

I promise this isn't just a ruse to get a free copy of Minecraft.  In fact I've somehow been able to keep myself from burning up hours and hours playing it, but please don't send me a free copy, my will is only so strong.  

Accessing localhost in Chrome when offline

I have been having a weird issue lately where I can't access my localhost development server with Chrome unless I have an internet connection, which doesn't make any sense because I shouldn't need a net connection to access my own machine, and I can get to it just fine in Firefox.  A little googling lead me to this Chrome bug that has an easy enough fix at the very bottom.  All you have to do is launch Chrome with this additional argument: "--enable-ipv6".

A Tale of Tethering

I love my Android phone, it's everything I need in a portable device.  I downloaded and listened to one of Andrew Warner's awesome Mixergy interviews on my way to work this morning.  Last night while my son and I were waiting for my daughter to get out of her ballet class I pulled up a movie on Youtube for him and he was good to go for the 40 minute wait.  I've got all of my gmail contacts organized and searchable at the touch of a button, and they are always synced to my account online, just in case.  I can go from snapping a picture or recording a video of my kids doing something cute to posting it to Facebook or Youtube in seconds, which the grandparents love.  

What I've been missing is tethering.  I had heard that there were several options available, some required rooting your phone, some required that you run their proprietary  program on your computer (Windows only, Mac support pending, Linux out of luck), and most of them cost money.  What I wanted was something that I could run without rooting my phone, that supports Linux, and is preferably free.  

Enter Proxoid, a little app that turns your Android phone into a proxy server.  I had this up and running on my Ubuntu laptop in minutes by following the detailed instructions found here.  It supports Windows, Mac, and Linux, and best of all it's free!  

I started writing myself a little How To text doc so I could remember how to connect next time I need it, but the steps are pretty easy now that I've got it all setup:

  1. Run Proxoid on my phone.
  2. On my phone go to Settings > Applications > Devleopment > Enable USB Debugging.
  3. On my computer run "adb forward tcp:8080 tcp:8080"
  4. Change my proxy settings to point to proxy server "localhost:8080"
  5. Profit!
Now I can jump online on my laptop when I'm out and about by tethering to my android.  

Bad Code Is Depressing

I've been depressed lately.  I haven't enjoyed coding on AppRabbit as much as I used to.  I sit down to flesh out the next cool feature on my list and I've got this nagging in the back of my mind.  I've got dirty code.  Not a lot of it, just a little, but it's pretty bad.  And when I look at the offending code, when I try to deal with it, it rears it's ugly head and I find myself unable to think straight, unable to formulate a good attack plan, and I shrink in fear and loathing.  

How did this devil spawn come to be?  How did I let it get this bad?  Ignorance, pure and simple.  I didn't know about Django's awesome formsets (and modelformsets) and so I rolled my own client side solution to validate an arbitrary number of child form items and when I save the parent form, submit them one at a time to the server via ajax.  It's a mess, a brittle, fragile, ugly mess, and while it does work it's the most common point of failure in the site.  But like a sea captan who's ship is taking on water, I just kept forging ahead, trying to make land.  Trying to get the other features done and hoping that this one would hold until I can come back around to look at it.  What I didn't realize was that this feature was bogging me down emotionally, blocking my creative flow and basically just depressing the crap out of me.

So the other day I decided to take a hard long look at this particular piece of code and try to figure out how to cut out the cancer and make the whole thing work right.  Part of the problem was that so much of my design for this part of the site was tied into the constraints I thought I had to work under, so cutting out those constraints meant I wasn't tied to that design anymore.  And while this is a good thing overall, it meant that I had to do design again for this part of the app, and I'm slow at design.  

But now that I'm knee deep in the new solution, writing *gasp* unit tests to insure that it works as expected and keeps working as expected, I find myself much happier, and I think the clouds are clearing and I'm coming out of my slump.  It's truly amazing how much this bad code was bringing me down, how much it was bugging me.  Now that I'm dealing with it, I wish I'd dealt with it a long time ago!  But as with most regrets, I wasn't ready before, I didn't know what to do, and I didn't know what I didn't know.  I'm just glad I've pulled myself out of this slump sooner than later, because it was really depressing.

Just Try It, You'll Like It

I had been putting off messing around with image manipulation in Python/Django because I thought it would be difficult or annoying to figure out and gain a full understanding of.  I'm not talking about complex image processing or anything, just thumbnail creation, watermarks, rotating, that sort of thing.  Turns pretty much as simple as "import PIL".  Throw in a bunch of really awesome libraries like sorl and it's almost literally a no brainer.  Now I'm kicking myself for not just diving in months ago so I could have used more image handling in my other projects.  

I also put off getting started with writing automated unit tests for my Django projects, for pretty much the same reason, I thought it would be difficult and I wanted to start with the easy stuff.  Again, it turns out that it is both easy to get started with and it would have made learning some of those easier things a little easier still, if I'd just taken the time to read a little bit about Django's unit testing capabilities and tried it out.  

So why did I wait?  What caused me to think that these things would be so difficult?  I think a big part of it comes from my background as an ASP.NET developer.  ASP.NET is great for a lot of things, and it helped me get my start in software development, and for that it will always hold a special place in my heart.  But image manipulation and unit testing in .NET both require a fair amount of effort to get started with, at least for me, because I learn by example and by trial and error.  Unit testing is especially difficult to do with ASP.NET due to the strong bindings between the Page object and the web server, it's hard to get those split up, especially if you're working with legacy code, which I was back in the day.  

But it would be too easy to blame my fear of just getting started on the technology, when in reality it is in human nature to fear the unknown.  There's a saying that goes something like "It's not fear of darkness that quickens our steps in the night, it's our own imagination's ability to fill that darkness with horrors."  I subconsciously made these new skills seem overly difficult to learn and understand by adding my own made up complexities and roadblocks.  Don't do this.  

Now you might say "Mark, how do I get over these subconscious fears if I don't even really know that they're there, holding me back?"  And my answer is: I don't know, but I can tell you the pattern that I've seen in myself.  I have to have a reason to do it, I can't just decide "You know what would be cool, automatic image thumbnail generation and caching in Django, I bet that would be fun to try out!"  I need a project that I can get excited about that requires that I figure that stuff out, than it's easy to be motivated to do it.

So if there's something that you've been meaning to try, or something that you know you should know about but haven't found the motivation to do it, find something that you want that doing this thing is a requirement for, than you won't have any choice but to just do it.

Who's Got Your Back?

It's been said many times and by many people that startups should have multiple founders, and there are plenty of existing blog posts about why, but for some of us that isn't the way things are working out. If you're going at it alone like me, it's important that you establish a solid support network, even if they aren't there by your side coding. For me, it's my wife, who doesn't know anything about programming except what she's picked up from our casual conversations. She's not an expert in business, finance, technology, or entrepreneurship. Her specialty is history, and she really knows her stuff there, but that doesn't help me all that much with getting my startup up and running. What she does have to offer is worth more to me than an extra programmer, web designer, business specialist, or any other area that I might hope to get some support from future cofounders (maybe that's you?). She offers me support and encouragement, she helps to focus me when I start drifting or become discouraged, and she tries to understand as much about what I'm doing as she can, even if she'll never be a techie.

The other day a friend asked me "so what's the difference between Python, Django, and Ruby on Rails? Which one is a framework and which one is a language?" My reply was "Let me call my wife, she can explain it to you." She prides herself on trying to be involved and really listen when I go off on a tangent about whatever I'm hacking on. She doesn't know how to program, but she knows the terminology, how things relate to each other, and most importantly what they have to do with my startup...err...our startup :-)

I know a lot of developers out there who's wives don't care about programming, technology, or what their husbands do at work. I count myself lucky to have a strong support system for my startup built into my relationship with my wife. I'm not suggesting that a wife is the only acceptable replacement for a fully contributing cofounder, but the idea of an outside support system is the key. Friends, parents, siblings, whatever you've got, try to tap into people who can get excited about what you're doing and give you a platform to bounce ideas off of, or lift you up when you're feeling down or things start to get rough.

Code I wrote a year ago makes me cringe, but maybe that's a good thing.

Someone that I follow on twitter (I can't find the tweet now, or remember who exactly it was) recently mentioned something about how looking at code that they wrote a year ago is painful.  I often feel the same way, although sometimes for me the timespan is much shorter, especially when I learn something new that ends up being a huge productivity boost (like how I recently started using virtualenv, even though I've been hearing about it for a long time).  I look back at some complex architecture that I setup to handle some equally complex business rules, only to see it in the new light of an additional year, month, or even week of knowledge and experience, and I can't help but immediately see better ways of doing it.  These epiphanies are often forehead-slap inducing.

I've heard a lot of other programmers talk about this learning cycle, and about how painful it sometimes is for them to confront their own handiwork months later.  It occurred to me that I usually only hear this type of lamentation from developers that are passionate about their work, at least on some level, and not so much form the entrenched code monkeys who are churning out CRUD all day and are ok with that.

So if my code continues to look good to me after a year, does that mean I'm not growing anymore, not progressing as a developer?  Maybe it just means that right now I'm doing a pretty good job and there isn't that much more for me to learn and improve on over the next year, but I find that hard to believe.  I've learned enough to know what I don't know, and there's still a lot that of it.  If anything, I'd like to see that window of recognition shorten, I'd like to be learning so much every day that my code ceases to meet my own expectations after a month, or even a week of new skills and experience.  Not to imply that I'm ok with writing crappy code, because I'm definitely not, I only hope that I can raise the bar so high, and so quickly, that my very best work this week will look like crap next week.