Where I work, our tool supports 6 different languages, where translations are contained in various sets of language files. One of these sets consists of the translations stored in Javascript files as JSON blocks, something like translations.en.js containing a structure

Translation works by looking for spans with an attribute data-lang, which in the above example would be <span data-lang="helloWord"></span>, and swaps in the relevant string to make it <span data-lang="helloWord">Hello world</span>.

In this particular instance, the current language was only obtainable from the client, so the page would have to load while var current_lang was initialised. Once I had the current language, I was looking to load the file on the fly, kind of like the way you add Google Analytics code to your page, something like (in <script> tags in the document head):

Further down the document, I had my window.onload function which swaps in the translations by looking up the Translations object.

As usual, this worked fine in all browsers, except Internet Explorer. You would expect that a browser’s window.onload event would only fire once all files have been loaded and the document has ‘settled down’. However, with IE, window.onload was firing before the language file had loaded and thus I was getting an “unknown variable: ‘Translations’” kind of error.

The Solution

In order to get around this, I created a global variable called translationsLoaded (set to false) and tied it to the window object, just before I append the language Javascript to the head, i.e. (from the example above)

Then, further down in my file where I have the window.onload I did:

Then, finally at the end of my translations.en.js I added code to set the variable to true and a manual call to the window.onload function:

Conculsion

It’s frustrating when IE (even modern IE8 and 9!) just don’t behave like all the other browser. Luckily, once you find the source of the problems, there’s usually some sort of work around.

This was all fine, but got very frustrating when dealing with arrays, objects and even strings with newline characters. The reason for this frustration was that newline characters were being displayed in the error log as \n and no new line, so something like

would produce:

instead of the nicer:

Not too bad for small arrays but for large objects it’s a nightmare! Googling around didn’t have an easy answer. I’m sure it’s some setting deep in php.ini or my Apache config, but I managed to come up with a pretty neat solution, which I’m going to document here.

I started by creating my own error log in /tmp/derror (David’s Error = derror!) and letting any one write to it (I realise this could be slightly cleaner/more secure):

Next I needed a nice debug function to write to this file. I wrote one that’ll take any number of parameters and for each one, output the current time along side a dump of their value. If it’s an object or class, use print_r, otherwise just display the value:

So, that function works fine but after a while I realised by doing my tail -f on my new error file, I was missing out on the real server errors. So, I had 2 different terminal tabs open, flipping between them both, which got annoying after about 2 errors! Luckily, I quickly thought of a way around this: route PHP’s error logging to my new file. The way Apache and virtual hosts are set-up this was quite easy and you can even log to 2 different files, thus preserving your ‘normal’ error file too. so, in your /etc/apache/sites-available/default or whatever, in the <Virtualhost:*.80> block for your site, copy the line for CustomLog and point that to your error file, i.e.

This should now route all Apache errors to your new file, so you have the best of both worlds!

To prevent the file getting to big over time, I set-up a cron job to simply delete the file and create a new one every Monday morning.

• params(consumer_key)
• generate_nonce(size)
• signature_base_string(method, uri, params)
• url_encode(string)
• sign(key, base_string)
• header(params)
• request_data(header, base_uri, method, post_data=nil)

The Log-in Process

When you sign up for an API key from Twitter, Yelp etc. you’ll be given a Consumer Key and a Consumer Secret. You may also be given an Access Token and an Access Token secret, but that’s for you logging into your own account. If that’s all you wish to do, you can skip this section.

So, if you want to let a user log in to their Twitter (or whatever) account via your site, you need to get an access token. The process for this is as follows:

1. You request a unique Request Token from Twitter for your ‘Log-in with Twitter’ button
2. You use this request token, as well as where on your site you want the user to be re-directed back to after they’re authenticated, to build the URL the button points to
3. They click the ‘Log-in with Twitter’ button on your site
4. The user is brought to Twitter where they enter their username and password
5. Twitter re-directs them back to your site, to the URL you gave them in step 2
6. Back at your site, you’ll now have an oAuth verifier
7. This can be used to get the user’s Twitter user info, which has been authenticated by Twitter

Step 1: Getting an oAuth request token for your current session

Twitter’s URL for getting a request token from is https://api.twitter.com/oauth/request_token. The request for the token contains a number of parameters, which are then combined with the URL you’re going to be sending the data to and the method of your request (i.e. GET, POST etc.) to generate what’s called a base signature. At this point you don’t have an access token, so your signing key is simply your consumer secret followed by an ‘&‘. Using the functions I’ve mentioned earlier, this can be done as follows:

Steps 2-5: Authenticating with Twitter

Once you have your request/access token, the URL to direct your user to is simply:

You can use standard <%= link_to 'Login', @login_url %> to generate a HTML anchor in your ERB template or whatever you choose. The user will then be directed to Twitter, where they enter their log-in details and get directed back to your callback URL.

Steps 6-7: Getting the user’s info

When the user is directed back to your site, you’ll be sent an oauth_verifier. This can be obtained via Rails in your callback URL’s corresponding controller, via params[:oauth_verifier]. You need to use this to request their user info, as a final check before they’re fully logged in. This results in a new auth token (or what Twitter calls an access token) and auth token secret, which should replace your previous stored values. It is assumed the code below is stored in a model class, where you need to pass the auth verifier from your controller.

data will now contain an array with things such as screen_name, user_id etc. of the user who just logged in. It’ll also contain a new oauth_token and oauth_token_secret, which should be saved as they’ll be used again.

Now you have a fully validated user, who has authenticated you to access information on Twitter via their Twitter account. So now, let’s access some of that info.

Getting the people a user is following (a GET request)

The process for all GET requests is pretty similar and roughly follows what we’ve done before. We have our array of standard parameters. To this, each of the GET parameters are passed. We use our access token and consumer key & secret to generate our oAuth signature, make the request and parse the response.

Following a specific user (a POST request)

The process for a POST is pretty similar, the only difference being how you handle the parameters to the request. In the example below, I’m assuming you have the user ID of the person you want to follow and that it’s stored in a variable called followee_user_id.

So, assuming that was successful, the user should now be following the person with user ID followee_user_id.

Conclusion

Hopefully this will fill in some of the gaps in Twitter’s documentation. When coding this I found plenty of instances where the documentation would say something like “now sign the key”, without actually telling you how to sign it! Very confusing indeed.

Disclaimer: I obviously wrote all this in proper Ruby on Rails classes and controllers, but have extracted the code out here to be presented in modular form. Thus, what’s here is not fully tested, or even elegant, but there should be enough to figure out what you need to do.

Each request will have a method (i.e. GET, POST etc.), a base URL to handle the request at the source site (Twitter, Yelp etc.) and a certain set of parameters. Every request I’ve dealt with has had the same 5 parameters, along with various other ones specific to the request you’re making. So, in Ruby, I’d have something like:

Next, you’ll need to add in any extra parameters to your params hash, e.g. your access token if you have it, and then combine all the above to generate a base string:

Next up, you need to generate a signing key, which is a combination of your consumer secret and your access token for the current session, if you have one at this stage (you may not, if the user still hasn’t logged in yet: in that case, a blank string will suffice). With this signing key, you sign your signature base string to get your oauth signature:

At this point, you’ve all your info nicely encoded in the oauth_signature using your private consumer secret. So, in a kind of public/private key partnership, you need to give the service your public consumer key, so it can validate the encoding of the oauth_signature at the destination:

So, you’re nearly ready to make your oAuth request. One final thing: all these parameters need to go into the Authorization line in your HTTP header, which is simply a matter of generating another string, as well as indicating you’re using oAuth:

So, to make your HTTP request, I wrote a generic function (request_data) that will do either a GET or a POST, make the request and return the response body:

And there you go. You should have a response from the API in whatever format you requested, be it JSON, XML or whatever. In my next post, I’ll explain how to use the above specifically for the Twitter API.

The process

Initial interview

First up, as with all jobs, you submit your resume/CV. This is then screened over by the development manager, who invites adequate candidates to a first-stage interview. This interview consists of him and another developer, basically going through the CV with the candidate, asking some general programming related questions, all just to try and get a better overview of the person, ensuring they pass the “no jerks” policy.

Programming Assignment

This is the main beauty of their hiring process, and is discussed in more details below. Candidates are given an assignment to work on in their own time at home, typically over a weekend. The assignment asks the user build a simple web form, with one text input to take free-text search strings that a user might enter when searching for property. They even give sample queries, such as “2 beds for sale in Dublin”. This string is to be submitted to a PHP script where all the necessary data is to be parsed from the string. Once parsed, a call is to be made to the Daft database via their API in order to get a list of results back. This result set is then displayed to the user. Finally, they need to write-up their code, explaining what’s going on in simple English.

Second Interview

Assuming you do a good job in the assignment, you’re then invited back for a more formal interview with senior management (company of about 60 people), to discuss career aspirations, salary requirements etc.

Why I think this is great

Skills it shows

As mentioned earlier, the key part of the process is the programming assignment. First up, the initial entry form and data submission. Here the user can show their knowledge of web security, CSS, (X)HTML standards and more. While none of these are actual requirements, it’s an easy place to show existing, fundamental web knowledge.

Once the data’s at the server, cleaned and verified, relevant data blocks (e.g. area, for sale/to let, number of bedrooms etc.) need to be parsed from the string. There are a number of different ways to do this and gives the developer plenty of scope for flair. A certain amount of analysis is required here also, mapping the domain of real estate to known keywords and looking for these in the string.

You also need to try and spot any relevant areas (e.g. County Dublin, Galway City etc.) the user may have entered. Without going into too much detail (in case potential candidates are reading this), there’s a key optimisation when doing this via the API and it’s a great way to spot those who have been programming seriously for a few years. Even if a programmer doesn’t implement this optimisation, they should still be aware of the problem it solves; this awareness can come across in their second interview, which is also accepted.

Once you know what the user is searching for, you then need to build a call to Daft’s SOAP API. This is good because it usually requires the candidate install PHP’s SOAP interface. While not overly complicated, it’s not trivial either and shows that the candidate is able to play around on a UNIX machine. Using the API also illustrates a candidates ability to follow documentation and quickly get to grips with a new system. Finally, by building a query from your parsed data that can be understood by the API, the candidate again has a chance to show some good, tidy code.

After pulling data from the API, the candidate again has a chance to show off XHTML/CSS skills to display the results set. Further ‘enhancements’ could be shown by doing all this over AJAX, without a page reload.

Finally, the candidate needs to document their code. This will illustrate communication skills, their attention to detail, writing skills (grammar, spelling) and more. Everyone in the software industry knows that communication is a large part of software development and it’s important to have good communicators on your team. Good code followed by an average write-up can often be less preferable than average code (which is generally easier to improve on) with a good write-up.

Why it’s better than others

While reading about other people’s interview experiences, one thing that stood out as annoying most developers is white-board coding, i.e. where you’re required to write some code on the fly, either on a white board or piece of paper. This a completely unrealistic situation and is asking a programmer to do something way outside their normal environment. Similar to this would be writing a program in the employer’s office in the space of an hour to 90 minutes; again this is wholly unrealistic as programmers are used to their own environment, have their own tools etc. The beauty of the Daft assignment is that it’s too tricky to do in less than two hours, plus they let you do it at home on your own machine. Yes, this does allow for potential plagiarism, but once you’re offered a job, you’re put on 6 months initial probation, so if you don’t code your assignment it’ll soon become obvious and they’ll have scope to not make you permanent.

Another great thing about it is that there’s no generic analytical questions such as “Why are man-holes round?” or “How many ties were sold in New York City last year?”. The value in asking these questions is surely waning these days, as they are easy enough to prepare for and people have learnt to expect them. Good programmers need to be good at analysis, but enough information about a candidates skills can be gleaned from how they approach the assignment.

Finally, there’s no over-bloated 7 round interview process. From the 2 interviews a candidate does, coupled with the assignment, it’s very easy to see what a person is like and that they’ve a decent level of programming competency.

One flaw

Not everything’s perfect and after chatting to my manager about the interview process, in preparation for this blog post, he pointed out that it’s poor at highlighting any “rockstar” programmers, as well as any existing systems administration skills. So, if you have 2 people competing against each other, one a decent programmer, the other amazing, it can be hard to differentiate between the 2; what’ll happen more often than not is that the ‘nicer’ one will be offered the job first.

Conclusion

From the above, it should be easy to see that current interview processes are far too complicated. 2 interviews will always be the least that’s required, but when you throw in a slightly complicated programming assignment, there shouldn’t be need for much more. The assignment’s a good way to test programming competency while the interviews are good to test a person’s likeability and career aspirations.

Error: Duplicate entry ’0′ for key 1

Clearly this had something to do with the primary key, which was a simple unique integer ID. As this is a pretty large table, I had a feeling that we had reached the upper limit of what could be stored for the field type (MySQL’s MEDIUMINT, signed). Looking at the maximum value for the ID, I saw it was 8388607; according to this table on the MySQL website, this value is the maximum that can be stored in that type of field, so this was clearly the problem. (N.B. an auto-incrementing field shouldn’t ever be defined as signed, as you’ll never go into the negative indices, but that was done before I came along!)

The solution? Surely a simple readjusting of the key to be unsigned, or to further future proofing, changing to the larger INT… That’s what I thought and quickly (in test first, of course!) did a

This should then change the maximum value allowed in the ID field to 16777215. So, to test this, I went to insert a row, specifying NULL for the primary key value. However, I still got the same error. Doing a DESC on the table told me that the ID field had been changed to MEDIUMINT UNSIGNED correctly, so that wasn’t the issue. After further research I determined that what happend was that MySQL’s internal counter for that auto increment field was still set to 0, due to the rollover caused after reaching the maximum integer value. To overcome this, you need to point the internal counter back to where it should be, i.e. the current maximum value of your ID field, as follows:

So, you set it to a value that’s one greater than the current maximum ID.

Haven’t posted for a while, mainly because I’ve been busy trying to teach myself Ruby on Rails, so haven’t created anything new in my spare time. Have come up with a few interesting fixes/ways to do things in relation to Facebook code in work though, so will hopefully do a post on those in the near future.
In the meantime, I wrote a very small little web application last week using the pretty cool jQuery Mobile framework. The app, called What’s on now (link removed, see UPDATE above), is simply a list of what’s on now and next on Ireland’s 17 basic cable channels. The data is pulled from a similar page on entertainment.ie, with the unnecessary channels filtered out. At the moment the app is pretty simple, but I plan to add to it over time (e.g. to add an option for the non-basic channels), updating this article as I go.

I did this mainly to have a quick go with jQuery Mobile, to see what it could do. I could’ve used PHP and built a mobile-sized HTML page, but it’s always good to try new things! Most of the development was pretty straight forward; however, because the data is retrieved dynamically every time the page is loaded, there’s a couple of tricks you need to apply to get the jQuery Mobile framework to do it’s magic.

The main page you’re greeted with is simply a glorified un-ordered list of programmes, with separator list items to distinguish the channels. I’m not going to go into the details of how you need to structure a page (see links at the end of this post) but here’s a snippet of the HTML:

When the page loads, ul#channels is empty. The data is called via a jQuery GET, which gets over cross-domain restrictions by using YUI, thanks to the Cross-Domain AJAX mod. The relevant data is filtered out and formatted and each of the li‘s are built and inserted into #channels. At this point, if you look at the page in your browser, it’ll still look like an ordinary list – you need to tell jQuery to work it’s magic on the dynamically created data. In this instance it’s done as follows:

Once I had my list of programmes, I thought I may as well add the info for each program, seeing as I already had the data at my disposal. The route I decided to go down here was to create new ‘page’ div‘s for each program, each one having it’s own ID, then link to each of these pages from the li‘s. Again, the scope of building one of these pages is beyond this blog post and well documented elsewhere, but here’s a quick sample:

This is simply added to the body using $('body').append(page); (where the variable page is a string of HTML such as the above). So, again here you need to tell jQuery Mobile that you’ve added a new page, so it can do it’s magic. This is achieved by the one simple line:

Hopefully this post will clear up a couple of things for anyone using jQuery Mobile with dynamically generated data. As I promised, here are some links of articles that helped me get a better understanding of the framework:

• http://msdn.microsoft.com/en-US/scriptjunkie/hh144955.aspx
• http://jquerymobile.com/demos/1.0a4.1/
• http://www.elated.com/articles/jquery-mobile-what-can-it-do-for-you/

Full implementation code (UPDATE 2)

I was requested by @danny to post the full source code, seeing as the app is actually no longer available, so I’ve decided to put most of it here. I excluded some of the scrape JS (as indicated in the code comments) to prevent the app being re-used somewhere else.

So, first up, is the initial HTML page, with header and footer blocks, an empty content block and the JS and CSS includes:

Next up is the javascript file, whatson.js in the above. Don’t forget, the $.ajax call has been overwritten by the Cross-Domain AJAX plug-in I mentioned earlier. Addtionally, I’ve used a few functions from php.js. to replicate this functionality in JS, namely str_replace, in_array and trim. I’ve excluded them here but they can be found in the php.js website.

Also, just to re-clarify, the page i was scraping had a list of channels and programs that were on now (prog1) and next (prog2).

I realise this code could be alot cleaner, but the app was still in it’s early stages before I was asked to take it down, thus I haven’t spent time tidying it up. Hopefully there’s enough here to figure out how to do what you need!

However, with older systems, especially those built in-house, you end up with the HTML entity version of certain characters in your database. It’s pretty much a fact of web development. Let’s use the example of a string that says “Price: €100″ but gets stored in the database as “Price: €100″. When you go to display this text on your encoded web-page, you end up seeing things such as “Price: €100″ in your browser. This is a result of double encoding, as the & in € is first getting encoded as &.

In order to remove these, I came up with the following function, that uses a simple regular expression to tidy such instances up.

What this does is looks for any 2 to 7 letter strings with & immediately before them and ; immediately after. When it finds a match, it simply replaces the & with &. It does this for all instances in your input string.

Update: Forgot that you can also have purely numeric codes here, so added ’0-9′ to the regex.

It all worked fine in most browsers, except of course Internet Explorer (IE). To be fair, IE8 was fine; however, seemingly at random, the zoom control would sometimes not appear on the map for IE7 and IE6 and you’d get one of IE’s very unhelpful error messages. At Daft we like to ensure that the site is available and consistent on all browsers, so it was imperative to get this issue sorted.

Because of the aforementioned unhelpful error messages, it proved to be a very difficult bug to nail down. I began to suspect that it was due to the code from Google loading after the map is set-up in the daft.ie hosted file, i.e. IE was calling Google-specific functions before it had the code. As it was all wrapped in a jQuery $(document).ready I didn’t think this could happen, but turns out that in the older versions of IE it could! I tried moving the JS around, loading the Daft-specific JS as late as possible, to try and give the Google JS the maximum amount of time to load, but that was to no avail.

So, I needed to find a way to load the Daft JS when I was 100% certain the Google JS had arrived. After various futile attempts, the solution ended up being pretty simple – change my

to

That way, the bulk of my JS is only executed once everything else has loaded and sure enough, the issue disappeared!

My suggestion would be to do a mod-rewrite (if using Apache, but other web servers have equivalents) on all incoming files to your 503 file, something like

If you put this rule first, you can leave the rest of your rules where they are as the “L” at the end of the line tells Apache that this is the last rule it needs to process, i.e. it can ignore the others.

Then, in your 503.php you need to tell any web-crawlers/site-indexers that the service is down but will be back at a certain time using PHP’s header function, as well as telling your users the same using a plain simple descriptive paragraph:

Should all be pretty straight-forward, but it’s good to let people know that you know you’re down and when you expect to be back.