Solr DataImportHandler issue: positive integers indexed for string ‘nested’ fields

A quick note about a Solr issue that took me some time to solve.

If this sounds familiar….

• You are using the DataImportHandler for Solr
• You have a entity with a field which values come from a related entity.
• After an import it looks like Solr only indexed even postive integers if you look at the schema browser.

….You probably have a ‘nested’ field which name is similar to it’s entity name. See the code below: entity name = regio and field name = regio. Changing field name to something else (regions) solved the issue. When you think about it, it’s somewhat logical that you don’t allow field names to have the same name as the entity. An schema exception during indexing would have been nice though.

Zend SOAP Server Webservice quickstart

Below a quick writeup of my first impression with building an basic Zend Soap Webservice. I invite you to add spelling and grammer corrections in the comments for my education.

Starting point

• My team needs to implement a SOAP service for mass posting of vacancies to a job board system.
• The SOAP service is based on a WSDL of an existing service. So we’ll use these specifications as a starting point for the proof of concept.
• On a site-note: I prefer REST above SOAP, because of it’s elegant simplicity. But it wouldn’t make a lot of business sense in this case because a lot of, paying, consumers of the new service have working code for the old service. Adapting to a slightly changed SOAP service will be much easier than a switch to a brand new REST API.

Available SOAP Server extensions for PHP

There are several frameworks / extensions / toolkits for creating a SOAP server for PHP:

• Pear SOAP package. Probably an orphan package as it’s not updated since 2008 and has a beta status. You probably want to look at the alternatives.
• NuSoap SOAP toolkit.  Started in 2002 and still under active development as the last release was just a few months ago at the time of writing.
• PHP 5 SOAP extensions. The official SOAP extension for PHP since version 5.
• Zend SOAP Server. Part of the Zend Framework, so probably not very useful if that’s not your current PHP framework.

As we use the Zend framework for this project, it was a natural choice to use it’s SOAP server implementation. We might opt for one of the alternatives if we slam into a brick wall later down the line.

Testing the waters

The steps I’ve taken to get a basic Zend SOAP Server based on the WSDL up and running

• I copied the wdsl to the /public directory of the Zend framework application making it publicly accessible under  http://example.org/jobtool.wsdl
• I created a new controller under application/controllers/soapController.php with an public indexAction function. Example code
• The new SOAP service is now available under http://example.org/soap
• Next step: actually handle SOAP requests. Example code. Handling of the soap request is as expected: SOAP method arguments are passed as function arguments. Complex types are represented as a stdClass objects, which basically are associative arrays. Nested complex types are translated to nested stdClass instances. You don’t get any warnings or exceptions if your argument count is different than specified in the SOAP request. IMHO that’s undesirable. I rather have big fat ugly exceptions in that case than subtle bugs.  The associative array you return are translated to the complex type as specified in the WSDL and returned to the client.
• To test the SOAP service without the need for a full-blown client i’ve used the free soapUI tool. You point this tool to the WDSL and it automatically creates fake soap request that you can use to test your brand new SOAP services. Make sure you specified the correct urls in the soapAction attributes in the WSDL.

Closing Thoughts

I hope this post saved you some when time building your first SOAP Webservice using Zend Framework. I don’t know yet from experience if the Zend SOAP Server will handle more advanced scenario’s. Only time will tell. Let me know how it works for you.

Simple ranked text search for MongoDB

In this code snippit you can see how to do a basic ranked text search for MongoDB. The code relies on two simple mapreduce operations. One to create an inverted index from some demo text, and a second one to score the matching documents based on query term hits.

MongoDB first impressions

For a customer we have developed log analytics software. It’s currently uses MYSQL as the database backend. The system reads in a hourly log file, and calculates all kinds of fancy statistics. I wanted to see how the system would work if I used MongoDB, a schema-less document DB, instead of MYSQL. My impressions in no particular order:

• Importing log data is much easier than on MYSQL because MongoDB is schema-less. Just create a collection (=bucket) and insert every log line into it as a hash. For log files that don’t have a fixed amount of fields, it’s a great fit.
• Like MYSQL, you do need to create indexes to make searching fast(er).
• MongoDB supports map reduce operations. It made some of the calculations much more elegant and better readable than the code that was written for MYSQL.
• Chaining of map reduce operations is supported, and works as you would expect.
• Queries are written in javascript. I’m happy that they didn’t invent yet another ’scripting’ language. Javascript looks capable enough.
• Map reduce operations are not particularly fast. They are upgrading their javascript engine to V8 to improve the execution speed.
• MongoDB community is nowhere near the size of MYSQL. Don’t expect a lot of Google results for a specific mongoDB issue. The moderated Google group is a better place to go currently.
• I liked the API. Calls are not verbose and their intented use is easy to understand.
• Although quite capable, mongoDB is still a young project. I need to have more time with it before using it on a customer project.

5 Must-have iPhone libraries

A list of  5 must-have iPhone libraries that we use at tty.nl for our iPhone dev needs:

#1 ASIHTTPRequest

The network API of Apple’s Cocoa Touch framework is not a pretty sight. It’s verbose and hard to put into use. The ASIHTTPRequest library by All-Seeing Interactive is high quality wrapper around the CFNetwork API that makes http communication trivial to implement. Love it!

#2 Json-framework

REST & JSON are here to stay. (Bye bye SOAP, won’t miss you!). The json-framework library enables your killer iPhone app to parse & generate strict JSON.

#3 Touchcode

Cocoa’s NSXML API is much too complex for the 95% of the use cases. For that 95% you should just use touchcode. It’s based on the Open Source libxml2 and supports most XPATH queries.

#4 Pinch analytics

Pinch analytics is best described as Google Analytics for your iPhone app. It allows you to track the amount of unique users, the time spent per user, device types and much more. And just like Google’s package it’s totally free.

#5 RegexKitLite

I was shocked to discover the Cocoa Touch framework doesn’t support Regular Expressions out of the box. No worries though: RegexKitLite allows you to continue to use your carefully crafted regular expressions.