Splitting a database dump

This is a way to split a SQL dump into tables that is relatively easy.

It should start with zcat FILE.gz | csplit -ftable – “/DROP TABLE/” {*},
but csplit has bug where reading a lot from standard in does not work.
So instead unzip your file first.

gunzip FILE.gz
csplit -ftable FILE "/DROP TABLE/" {*}

Then to give the files meaningful names:

for FILE in `ls -1 table*`; do
     NAME=`head -n1 $FILE | cut -d$'\x60' -f2`
     mv $FILE "$NAME.sql";
done;

If your dump does not start witjh a DROP TABLE `name` IF EXISTS,
you will have to change the match expression to csplit a litte.

Hope this is useful to someone.

Setting up Emacs to compare two git tags

Introduction

This is a setup for using Emacs to compare two git tags. It is based on Ediff mode and Ediff Trees. Ediff Trees is a useful front-end for comparing large trees of files. To make things easier we have two git clones, one for each tag that we call before and after.

Needed Code

Create a file named ediff-trees.el with the following contents and place it in your Emacs load path:

;;; ediff-trees.el --- Recursively ediff two directory trees
;;;----------------------------------------------------------------------
;; Author: Joao Cachopo <joao.cachopo@inesc-id.pt>
;; Created on: Wed May 10 17:30:49 2006
;; Keywords: ediff, comparing
;; Version: 20071126.1
;;
;; Copyright (C) 2006 Joao Cachopo

;; This program is not part of GNU Emacs

;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License as
;; published by the Free Software Foundation; either version 2, or (at
;; your option) any later version.

;; This program is distributed in the hope that it will be useful, but
;; WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, 675 Massachusettes Ave, Cambridge, MA
;; 02139, USA.

;;; Commentary:

;; The ediff-trees package is a simple frontend to the emacs' ediff
;; package to allow a simpler comparison of two similar directory
;; trees.

;; I wrote this package because I often need to compare two different
;; versions of the same directory tree and ediff-directories is not
;; very helpful in this case.  Specially when the directory trees to
;; compare are deep and only a few files have changed.
;; Typically, that occurs when I create a copy of some project
;; directory tree either to make some experiments myself or to send to
;; someone else that will return a modified directory tree to me
;; later.  (Yes, I heard of version control systems, and I use them
;; regularly.  Yet, for several reasons, sometimes that is not an
;; option.)

;; Later, when I want to integrate the modified directory tree with
;; the original tree, I want to see the differences to the original
;; version, so that I may decide whether to accept the changes or not.
;; This is where this package kicks in...

;; To use it, just call `ediff-trees', which will ask for two
;; directories to compare.  Usually, I give the original directory as
;; the first one and the modified directory as the second one.

;; ediff-trees recursively descends both directories, collecting the
;; pairs of files that are worth "comparing": either files that
;; changed, or that appear in one of the two directory trees but not
;; in the other.  Then, it shows the first "change" using ediff.

;; In fact, ediff-trees either uses ediff to compare a file with its
;; changed version, or simply opens a file that occurs in only one of
;; the trees.

;; The user can then navigate backward and forward in the set of
;; changes by using `ediff-trees-examine-next' and
;; `ediff-trees-examine-previous', respectively.  These functions move
;; from one change (quiting the current ediff session or killing the
;; current file buffer) to another.  Therefore, by repeatedly using
;; these functions we can go through all the changes.  I usually use
;; some global bindings for these functions.  Something like this:
;;
;;   (global-set-key (kbd "s-SPC") 'ediff-trees-examine-next)
;;   (global-set-key (kbd "S-s-SPC") 'ediff-trees-examine-previous)
;;   (global-set-key (kbd "C-s-SPC") 'ediff-trees-examine-next-regexp)
;;   (global-set-key (kbd "C-S-s-SPC") 'ediff-trees-examine-previous-regexp))

;; The `ediff-trees-examine-next-regexp' and
;; `ediff-trees-examine-previous-regexp' skip over the list of changes
;; to a file with a filename that matches a given regexp.

;; This package allows for some customization.  Please, see the
;; ediff-trees group under customize.

;; Finally, to deal with small changes in the white space I often find
;; it useful to configure ediff like this:
;;
;;   (setq ediff-diff-options "-w")
;;   (setq-default ediff-ignore-similar-regions t)

;;; Code:


(require 'ediff)

(defgroup ediff-trees nil
  "Extend ediff to allow comparing two trees recursively."
  :tag "Ediff Trees"
  :group 'ediff)


(defface ediff-trees-deleted-original-face
  '((((class color))
     (:background "Pink"))
    (t (:inverse-video t)))
  "Face for highlighting the buffer when it was deleted from the original tree."
  :group 'ediff-trees)

(defcustom ediff-trees-file-ignore-regexp
  "\\`\\(\\.?#.*\\|.*,v\\|.*~\\|CVS\\|_darcs\\)\\'"
  "A regexp matching either files or directories to be ignored
when comparing two trees.  If a directory matches the regexp,
then its contents is not scanned by `ediff-trees'."
  :type 'regexp
  :group 'ediff-trees)


(defcustom ediff-trees-sort-order-regexps nil
  "*Specifies a list of regexps that determine the order in which
files will be presented during the ediff-trees session.  Files
with filenames matching former regexps appear earlier in the
session.  If a filename matches more than one regexp, the first
one wins."
  :type '(repeat regexp)
  :group 'ediff-trees)



(defun ediff-trees (root1 root2)
  "Starts a new ediff session that recursively compares two
trees."
  (interactive
   (let ((dir-A (ediff-get-default-directory-name))
         f)
     (list (setq f (ediff-read-file-name "Directory A to compare:" dir-A nil))
       (ediff-read-file-name "Directory B to compare:"
                 (if ediff-use-last-dir
                     ediff-last-dir-B
                   (ediff-strip-last-dir f))
                 nil))))
  (ediff-trees-internal root1 root2))


;;; Internal variables, used during an ediff-trees session
(defvar ediff-trees-current-file nil)
(defvar ediff-trees-remaining-files (list))
(defvar ediff-trees-examined-files (list))


(defun ediff-trees-internal (root1 root2)
  (let ((files-changed (ediff-trees-collect-files root1 root2)))
    (if (not (null files-changed))
        (progn
          (setq ediff-trees-remaining-files files-changed)
          (setq ediff-trees-examined-files (list))
          (ediff-trees-examine-next 1))
      (message "There are no changes between the trees!"))))

(defun ediff-trees-collect-files (root1 root2)
  (ediff-trees-sort-files
   (nconc (ediff-trees-collect-changed-files root1 root2)
          (mapcar (lambda (el) (cons el nil))
                  (ediff-trees-collect-new-files root1 root2))
          (mapcar (lambda (el) (cons nil el))
                  (ediff-trees-collect-new-files root2 root1)))))


(defun ediff-trees-sort-files (files)
  (let ((tagged-files (mapcar (lambda (pair)
                                (cons (ediff-trees-get-sort-order (or (car pair) (cdr pair)))
                                      pair))
                              files)))
    (mapcar #'cdr
            (sort tagged-files
                  (lambda (tf1 tf2)
                    (let ((order1 (car tf1))
                          (order2 (car tf2)))
                      (or (< order1 order2)
                          (and (= order1 order2)
                               (let ((el1 (or (cadr tf1) (cddr tf1)))
                                     (el2 (or (cadr tf2) (cddr tf2))))
                                 (string< el1 el2))))))))))


(defun ediff-trees-get-sort-order (pathname)
  (let ((order 0)
        (sorting-regexps ediff-trees-sort-order-regexps))
    (while (and (not (null sorting-regexps))
                (not (string-match (pop sorting-regexps) pathname)))
      (setq order (+ order 1)))
    order))



(defun ediff-trees-collect-changed-files (root1 root2)
  (let ((changed (list)))
    (dolist (filename (directory-files root1))
      (unless (ediff-trees-skip-file-p filename)
        (let ((file1 (expand-file-name filename root1))
              (file2 (expand-file-name filename root2)))
          (when (and (file-exists-p file1) (file-exists-p file2))
            (if (eql (file-directory-p file1)
                     (file-directory-p file2))
                (cond ((file-directory-p file1)
                       (setq changed (nconc changed (ediff-trees-collect-changed-files file1 file2))))
                      ((not (ediff-same-file-contents file1 file2))
                       (push (cons file1 file2) changed)))
              (let ((msg (format "I cannot compare a directory, '%s', with a file.  Continue? "
                                 (if (file-directory-p file1) file1 file2))))
                (if (not (y-or-n-p msg))
                    (error "Aborting ediff-trees"))))))))
    changed))


(defun ediff-trees-collect-new-files (root1 root2)
  "Collect files from root1 that do not appear at root2."
  (let ((new-files (list)))
    (dolist (filename (directory-files root1))
      (unless (ediff-trees-skip-file-p filename)
        (let ((file1 (expand-file-name filename root1))
              (file2 (and root2 (expand-file-name filename root2))))
          (when (file-exists-p file1)
            (cond ((file-directory-p file1)
                   (setq new-files
                         (nconc new-files
                                (ediff-trees-collect-new-files file1
                                                               (and (stringp file2)
                                                                    (file-directory-p file2)
                                                                    file2)))))
                  ((or (null file2) (not (file-exists-p file2)))
                   (push file1 new-files)))))))
    new-files))

(defun ediff-trees-skip-file-p (filename)
  ;; always ignore . and ..
  (or (string= filename ".")
      (string= filename "..")
      (string-match ediff-trees-file-ignore-regexp filename)))


(defun ediff-trees-examine-next (num)
  (interactive "p")
  (if (< num 0)
    (ediff-trees-examine-previous (- num))
    (ediff-trees-examine-file
     (lambda (file) (zerop (setq num (- num 1))))
     (lambda (file) (push file ediff-trees-examined-files))
     (lambda () (pop ediff-trees-remaining-files)))))


(defun ediff-trees-examine-previous (num)
  (interactive "p")
  (if (< num 0)
    (ediff-trees-examine-next (- num))
    (ediff-trees-examine-file
     (lambda (file) (zerop (setq num (- num 1))))
     (lambda (file) (push file ediff-trees-remaining-files))
     (lambda () (pop ediff-trees-examined-files)))))


(defun ediff-trees-examine-next-regexp (regexp)
  (interactive "sSearch for (regexp): ")
  (ediff-trees-examine-file
   (lambda (file) (string-match regexp (or (car file) (cdr file))))
   (lambda (file) (push file ediff-trees-examined-files))
   (lambda () (pop ediff-trees-remaining-files))))


(defun ediff-trees-examine-previous-regexp (regexp)
  (interactive "sSearch for (regexp): ")
  (ediff-trees-examine-file
   (lambda (file) (string-match regexp (or (car file) (cdr file))))
   (lambda (file) (push file ediff-trees-remaining-files))
   (lambda () (pop ediff-trees-examined-files))))


(defun ediff-trees-examine-file (pred save-current-file-fn get-next-file-fn)
  (when (eq (current-buffer) ediff-control-buffer)
    (ediff-really-quit nil))
  (unless (null ediff-trees-current-file)
    (funcall save-current-file-fn ediff-trees-current-file)
    (when (car ediff-trees-current-file)
      (kill-buffer (find-buffer-visiting (car ediff-trees-current-file))))
    (when (cdr ediff-trees-current-file)
      (kill-buffer (find-buffer-visiting (cdr ediff-trees-current-file))))
    (setq ediff-trees-current-file nil))
  (let ((next-file (ediff-trees-get-next-file pred save-current-file-fn get-next-file-fn)))
    (if (null next-file)
        (message "No more files.")
      (progn
        (setq ediff-trees-current-file next-file)
        (if (and (car next-file) (cdr next-file))
            (ediff-files (car next-file) (cdr next-file))
          (progn
            (delete-other-windows)
            (find-file-read-only (or (car next-file) (cdr next-file)))
            (when (null (cdr next-file))
              (let ((overlay (make-overlay 0 (point-max))))
                (overlay-put overlay 'face 'ediff-trees-deleted-original-face)))))))))


(defun ediff-trees-get-next-file (pred save-current-file-fn get-next-file-fn)
  (let ((return-value 'not-found))
    (while (eq return-value 'not-found)
      (let ((next-file (funcall get-next-file-fn)))
        (cond ((null next-file)
               (setq return-value nil))
              ((funcall pred next-file)
               (setq return-value next-file))
              (t
               (funcall save-current-file-fn next-file)))))
    return-value))


(provide 'ediff-trees)

This code can be downloaded from the EmacsWiki. We made a change in the two functions ediff-trees-collect-changed-files and ediff-trees-collect-new-files where we added condition (file-exists-p file1) to skip symbolic links that do not refer to an existing file.

Create another file with the following contents, place it in your Emacs load path and load it:

(assert (and (boundp 'ediff-git-root-before) (stringp ediff-git-root-before)))
(assert (and (boundp 'ediff-git-root-after)  (stringp ediff-git-root-after)))

(require 'ediff-trees)

(setq-default ediff-ignore-similar-regions t)
(setq-default ediff-split-window-function 'split-window-horizontally)
(setq-default ediff-trees-file-ignore-regexp "^[.]?#\\|~$\\|^[.]git$")

(defun next-error-capable-buffer () "Return a 'next-error' capable buffer."
  (ignore-errors (next-error-find-buffer))
)

(defun kill-all-next-error-capable-buffers () "Kill all 'next-error' capable buffers."
  (interactive)
  (let ((buffer (next-error-capable-buffer)))
    (when buffer
      (message "Killing buffer '%s'" (buffer-name buffer))
      (kill-buffer buffer)
      (kill-all-next-error-capable-buffers)
    )
  )
)

(defun ediff-git-commits (commits) "Start a new ediff session that recursively compares 'before' and 'after'."
  (interactive "sEnter commit(s): ")
  (let* ((whitespace-chars " \f\n\r\t")
         (cc-whitespace (concat "[" whitespace-chars "]"))
         (re-commit (concat "\\([^" whitespace-chars "]+\\)"))
        )
    (if (string-match (concat "\\`" cc-whitespace "*" re-commit "\\(?:" cc-whitespace "+" re-commit "\\)?" cc-whitespace "*" "\\'") commits)
      (let ((commit-before (match-string 1 commits)) (commit-after (match-string 2 commits)))
        (unless commit-after
          (if (y-or-n-p (format "Compare %s with parent? " commit-before))
            (setq commit-after commit-before commit-before (concat commit-after "^"))
            (let ((commit (read-string "Enter commit for 'after': ")))
              (if (string-match (concat "\\`" cc-whitespace "*" re-commit cc-whitespace "*" "\\'") commit)
                (setq commit-after (match-string 1 commit))
                (error "Not a valid commit")
              )
            )
          )
        )
        (let* ((case-fold-search nil) (ok "ok") (re-command-ok (concat cc-whitespace (regexp-quote ok) "\\'"))
               (command-template (concat "unset CDPATH && cd %s && git checkout . && git fetch && git checkout %s && echo -n " ok))
               (command-before (format command-template ediff-git-root-before commit-before))
               (command-after  (format command-template ediff-git-root-after commit-after))
              )
          (message "Checking out 'before'...")
          (unless (string-match re-command-ok (shell-command-to-string command-before))
            (error "Error checking out 'before' (%s)" command-before)
          )
          (message "Checking out 'after'...")
          (unless (string-match re-command-ok (shell-command-to-string command-after))
            (error "Error checking out 'after' (%s)" command-after)
          )
        )
      )
      (unless (and (string-equal commits "") (y-or-n-p "No commit(s) supplied. Keep current checkouts? "))
        (error "No valid commit(s) supplied")
      )
    )
    (kill-all-next-error-capable-buffers)
    (message "Comparing...")
    (ediff-trees ediff-git-root-before ediff-git-root-after)
  )
)

(defun visit-next (arg) "In an ediff session, visit next file, else visit next 'next-error' message."
  (interactive "p")
  (if (or (next-error-capable-buffer) (null ediff-trees-current-file))
    (next-error arg)
    (ediff-trees-examine-next arg)
  )
)

(defun visit-previous (arg) "In an ediff session, visit previous file, else visit previous 'next-error' message."
  (interactive "p")
  (if (or (next-error-capable-buffer) (null ediff-trees-current-file))
    (previous-error arg)
    (ediff-trees-examine-previous arg)
  )
)

Make sure you have constants ediff-git-root-before and ediff-git-root-after defined in your config.
Example:

(defconst ediff-git-root-before "/home/eduard/before")
(defconst ediff-git-root-after "/home/eduard/after")

Usage

To start a new ediff session, execute ediff-git-commits (M-x ediff-git-commits) and enter zero, one or two commits. When comparing is done, execute visit-next(or bind it to a key you like), to go to the next differing files.

Harro & Eduard

Perl Quiz, Perl Solution: Plusified Equations

For problem definition see: The Haskell solution.

Because this was a test for an applicant, I felt I should solve it too in order to get a good idea of how difficult it is.

My approach was so generate the plussified expressions from smallest to largest sum. That way I can discard any expressions from one list that have a smaller sum then the least sum of the other list.

The Algorithm

  1. Generate a list of plussified expressions for each of the integers, sorted smallest to largest sum.
  2. Compare the sums of the first elements of both lists. If one is less then the other remove it and repeat.
  3. If the sums are equal you have found an answer.

    You could now drop the first expression of each list and continue,
    but you would miss the answers where you link the current first element of one list with
    expressions in the other list that are not first, but have the same sum.
    Therefore I also explicitly look for those answers with a recursive call.

Questions welcome at eduard@tty.nl .

#!/usr/bin/env perl
use strict;
use warnings;

main( @ARGV );

sub main {
    /^\d+$/ or die("'$_' is not a valid string of digits.") for @_;
    2 == @_ or die("Supply exactly 2 strings of digits seperated by a space");

    my @answers = find_answers( [ plusify(shift) ], [ plusify(shift) ] );
    print join("\n", @answers ), "\nFound " . scalar(@answers) . " answers.\n";

    return;
}

sub plusify  {
    my ($num) = @_;
    my ($first_digit, $rest) = $num =~ m/(\d)(\d+)?/;
    return ($first_digit) if not defined $rest;
    my @ways = map {  $first_digit . $_ , $first_digit . '+' . $_ } plusify($rest);

    return sort { sum_expression($a) <=> sum_expression($b) } @ways;
}


sub sum_expression  {
    my ($expression) = @_;
    my $sum = 0;
    $sum += $_ for split(qr{\+}, $expression);

    return $sum;
}

sub find_answers  {
    my ( $expressions_1, $expressions_2 ) = @_;
    my @expressions_1 = @{ $expressions_1 };
    my @expressions_2 = @{ $expressions_2 };
    my @answers = ();
    while ( @expressions_1 and @expressions_2 ) {
        my $head_1 = shift(@expressions_1);
        my $head_2 = shift(@expressions_2);
        my $sum_1 = sum_expression($head_1);
        my $sum_2 = sum_expression($head_2);
        if ( $sum_1 == $sum_2 ) {
            push(@answers,"$head_1 = $sum_1 = $head_2" ) ;
            push(@answers, find_answers( [ $head_1 ],  [ @expressions_2 ] ) );
            push(@answers, find_answers( [ @expressions_1 ], [ $head_2 ] ) );
        };

        # Effectively only shift the lowest of the heads
        unshift(@expressions_2, $head_2) if $sum_1 < $sum_2;
        unshift(@expressions_1, $head_1) if $sum_2 < $sum_1;
    };

    return @answers;
}

1;