How to fix 404 on Wordpress Permalinks on IIS

I wrote an article on my personal blog about fixing 404’s on Wordpress Permalinks on an IIS 6.0 on a Windows webserver. There were two different sources I got the 404 from and which I had to fix; first from Wordpress and then (Murphy’s law working here ;)) also from IIS. I hope it can help you if you have the same or a similar Wordpress problem.

Now at TTY we prefer setting up systems with open source software and therefor using servers with a Linux flavour and an open source HTTP server, but there may be situations where a Windows and IIS system is already in use. This article shows we are capable of handling just that.

Read the article here

Solr DataImportHandler issue: positive integers indexed for string ‘nested’ fields

A quick note about a Solr issue that took me some time to solve.

If this sounds familiar….

• You are using the DataImportHandler for Solr
• You have a entity with a field which values come from a related entity.
• After an import it looks like Solr only indexed even postive integers if you look at the schema browser.

….You probably have a ‘nested’ field which name is similar to it’s entity name. See the code below: entity name = regio and field name = regio. Changing field name to something else (regions) solved the issue. When you think about it, it’s somewhat logical that you don’t allow field names to have the same name as the entity. An schema exception during indexing would have been nice though.

Zend SOAP Server Webservice quickstart

Below a quick writeup of my first impression with building an basic Zend Soap Webservice. I invite you to add spelling and grammer corrections in the comments for my education.

Starting point

• My team needs to implement a SOAP service for mass posting of vacancies to a job board system.
• The SOAP service is based on a WSDL of an existing service. So we’ll use these specifications as a starting point for the proof of concept.
• On a site-note: I prefer REST above SOAP, because of it’s elegant simplicity. But it wouldn’t make a lot of business sense in this case because a lot of, paying, consumers of the new service have working code for the old service. Adapting to a slightly changed SOAP service will be much easier than a switch to a brand new REST API.

Available SOAP Server extensions for PHP

There are several frameworks / extensions / toolkits for creating a SOAP server for PHP:

• Pear SOAP package. Probably an orphan package as it’s not updated since 2008 and has a beta status. You probably want to look at the alternatives.
• NuSoap SOAP toolkit.  Started in 2002 and still under active development as the last release was just a few months ago at the time of writing.
• PHP 5 SOAP extensions. The official SOAP extension for PHP since version 5.
• Zend SOAP Server. Part of the Zend Framework, so probably not very useful if that’s not your current PHP framework.

As we use the Zend framework for this project, it was a natural choice to use it’s SOAP server implementation. We might opt for one of the alternatives if we slam into a brick wall later down the line.

Testing the waters

The steps I’ve taken to get a basic Zend SOAP Server based on the WSDL up and running

• I copied the wdsl to the /public directory of the Zend framework application making it publicly accessible under  http://example.org/jobtool.wsdl
• I created a new controller under application/controllers/soapController.php with an public indexAction function. Example code
• The new SOAP service is now available under http://example.org/soap
• Next step: actually handle SOAP requests. Example code. Handling of the soap request is as expected: SOAP method arguments are passed as function arguments. Complex types are represented as a stdClass objects, which basically are associative arrays. Nested complex types are translated to nested stdClass instances. You don’t get any warnings or exceptions if your argument count is different than specified in the SOAP request. IMHO that’s undesirable. I rather have big fat ugly exceptions in that case than subtle bugs.  The associative array you return are translated to the complex type as specified in the WSDL and returned to the client.
• To test the SOAP service without the need for a full-blown client i’ve used the free soapUI tool. You point this tool to the WDSL and it automatically creates fake soap request that you can use to test your brand new SOAP services. Make sure you specified the correct urls in the soapAction attributes in the WSDL.

Closing Thoughts

I hope this post saved you some when time building your first SOAP Webservice using Zend Framework. I don’t know yet from experience if the Zend SOAP Server will handle more advanced scenario’s. Only time will tell. Let me know how it works for you.

A different approach to web form design

An interesting article about a different way to look at forms: “Mad Libs” Style Form Increases Conversion 25-40%.

Conclusion of the article seems to be that conversion increases a lot when presenting the form as a narrative. Don’t know if that really is supported by the small test they did, but I’m inclined to think they might be on to something.

NLTK’s dispersion_plot on Mac OS X

While reading “Natural Language Processing with Python” I ran into problems on my Mac with examples that were using the dispersion_plot function: calls to the function returned immediately without displaying anything.

Turns out matplotlib’s back-end wasn’t configured properly. To fix this I had to add a rc file (matplotlibrc) to my ~/.matplotlib directory. The rc file contains the following:

And, hey presto:

(disclaimer: “Works on my machine!”)

A distributed index setup for Sphinx

Sphinx search is a powerful search engine. Recently we released it (version 0.9.9-rc2) as the backend for most of the searches on one of our high-volume websites. This site has about 360.000 visitors a day that generate about 4.500 search queries for the Sphinx backend per minute on average, peaking to nearly 9.000 per minute when it gets busy on the site. To be able to handle that many requests we currently run Sphinx on four dedicated servers.

A problem with having more than one sphinx server is that you need to make sure the results from the different server are close to the same. Since it is possible to switch between servers for two consecutive searches (which on the site in question could also be a browsing action, for example moving from one page of results to the next) it could be very confusing if the search result were different.

With Sphinx there are a number of ways to solve this problem. The most commonly used solutions are:

• run the indexer on one server and make those indexing results available to all the other servers (through scp, rsync, or hosting on a shared filesystem)
• using a distributed index setup

The first should work, but is actually not recommended by the makers of Sphinx. We went for the second solution: a distributed index setup.
Read the rest of this entry »

Simple ranked text search for MongoDB

In this code snippit you can see how to do a basic ranked text search for MongoDB. The code relies on two simple mapreduce operations. One to create an inverted index from some demo text, and a second one to score the matching documents based on query term hits.

MongoDB first impressions

For a customer we have developed log analytics software. It’s currently uses MYSQL as the database backend. The system reads in a hourly log file, and calculates all kinds of fancy statistics. I wanted to see how the system would work if I used MongoDB, a schema-less document DB, instead of MYSQL. My impressions in no particular order:

• Importing log data is much easier than on MYSQL because MongoDB is schema-less. Just create a collection (=bucket) and insert every log line into it as a hash. For log files that don’t have a fixed amount of fields, it’s a great fit.
• Like MYSQL, you do need to create indexes to make searching fast(er).
• MongoDB supports map reduce operations. It made some of the calculations much more elegant and better readable than the code that was written for MYSQL.
• Chaining of map reduce operations is supported, and works as you would expect.
• Queries are written in javascript. I’m happy that they didn’t invent yet another ’scripting’ language. Javascript looks capable enough.
• Map reduce operations are not particularly fast. They are upgrading their javascript engine to V8 to improve the execution speed.
• MongoDB community is nowhere near the size of MYSQL. Don’t expect a lot of Google results for a specific mongoDB issue. The moderated Google group is a better place to go currently.
• I liked the API. Calls are not verbose and their intented use is easy to understand.
• Although quite capable, mongoDB is still a young project. I need to have more time with it before using it on a customer project.

My Reading List for 2010

One of the suggestions of “The Pragmatic Programmer” is that you should learn at least one new programming language every year. This is a great suggestion, but after a couple of years its usefulness diminishes, e.g. if one already knows Perl and Python, then the payback on learning Ruby is rather small. Therefore I’m going to concentrate on the foundations of programming languages this year. Here’s my tentative reading list:

• Design Concepts in Programming Languages
• Concepts, Techniques, and Models of Computer Programming
• Types and Programming Languages

Suggestions welcome.

Ruby Quiz, Haskell Solution: LCD Numbers

A solution to Ruby Quiz #14 in literate Haskell:

LCD Numbers
===========

Problem
-------

[original source](http://rubyquiz.com/quiz14.html)

This week's quiz is to write a program that displays LCD style numbers
at adjustable sizes.

The digits to be displayed will be passed as an argument to the
program. Size should be controlled with the command-line option -s
follow up by a positive integer. The default value for -s is 2.

For example, if your program is called with:

    $ lcd.rb 012345

The correct display is:

     --        --   --        --
    |  |    |    |    | |  | |
    |  |    |    |    | |  | |
               --   --   --   --
    |  |    | |       |    |    |
    |  |    | |       |    |    |
     --        --   --        -- 

And for:

    $ lcd.rb -s 1 6789

Your program should print:

     -   -   -   -
    |     | | | | |
     -       -   -
    | |   | | |   |
     -       -   - 

Note the single column of space between digits in both examples. For
other values of -s, simply lengthen the - and | bars.

Solution
--------

Module declaration and imports:

> module Main where
>
> import Data.Char (digitToInt)
> import Data.List (intersperse)
> import System.Console.GetOpt
> import System.Environment (getArgs)

First we define the numbers at size 1:

> n0 = [ " - "
>      , "| |"
>      , "   "
>      , "| |"
>      , " - "
>      ]
>
> n1 = [ "   "
>      , "  |"
>      , "   "
>      , "  |"
>      , "   "
>      ]
>
> n2 = [ " - "
>      , "  |"
>      , " - "
>      , "|  "
>      , " - "
>      ]
>
> n3 = [ " - "
>      , "  |"
>      , " - "
>      , "  |"
>      , " - "
>      ]
>
> n4 = [ "   "
>      , "| |"
>      , " - "
>      , "  |"
>      , "   "
>      ]
>
> n5 = [ " - "
>      , "|  "
>      , " - "
>      , "  |"
>      , " - "
>      ]
>
> n6 = [ " - "
>      , "|  "
>      , " - "
>      , "| |"
>      , " - "
>      ]
>
> n7 = [ " - "
>      , "  |"
>      , "   "
>      , "  |"
>      , "   "
>      ]
>
> n8 = [ " - "
>      , "| |"
>      , " - "
>      , "| |"
>      , " - "
>      ]
>
> n9 = [ " - "
>      , "| |"
>      , " - "
>      , "  |"
>      , " - "
>      ]
>

Put the numbers in  a list:

> numbers = [n0,n1,n2,n3,n4,n5,n6,n7,n8,n9]

Horizontal scaling function, given a string replicate the second
character n times:

> hscale n cs = head cs : replicate n (cs!!1) ++ [last cs]

Vertical scaling function, repeat the second and fourth row n times:

> vscale n css = head css : replicate n cs1 ++ [cs2] ++ replicate n cs3 ++ [cs4]
>   where cs1 = css !! 1
>         cs2 = css !! 2
>         cs3 = css !! 3
>         cs4 = last css

Scale function; note this function scales a single number:

> scale n = vscale n . map (hscale n)

Function that converts a list of numbers to a string of LCD numbers:

> lcd n = concat .
>         intersperse "\n" .
>         foldr1 (zipWith (++)) .
>         intersperse (replicate (3 + 2*n) " ") .
>         map (scale n . (numbers !!))

`main` function:

> main = do
>   args <- getArgs
>   let (n, digits) = parseArgs args
>   putStrLn $ lcd n $ map digitToInt digits

Command-line argument parsing:

> data Flag = Scale Int
>             deriving Eq
>
> options = [Option "s" [] (ReqArg (Scale . read) "") ""]
>
> parseArgs args =
>   case parse args of
>    (_, [], _)              -> error "Usage: lcd [-s n] digits"
>    ([], digits, [])        -> (2, head digits)
>    ([Scale n], digits, []) -> (n, head digits)
>    (_, _, _)               -> error "Usage: lcd [-s n] digits"
>   where
>     parse = getOpt RequireOrder options