1. Aggregate your data and output it in some kind of structured format, usually XML
2. Request this information and generate some visualizations of it
3. Serve these visualizations through a web server

Pretty basic stuff. All of the tools that I have seen essentially make one or all of these processes easier. Unfortunately no matter what tools you choose, the rule of engineering stands: Fast, cheap, good – choose two

In this context it is more like: Less proprietary code, free software, easy to use/maintain – choose two. The good news is that all of the solutions described essentially have the same result: a web dashboard. The choice that needs to be made, is where you want to go with it? In the spirit of Enterprise software the clear winner is the solution with NO proprietary code, easy to use interfaces, and maybe some support.The biggest obstacle in this model is step 1, data aggregation and standardization.

Step 1 may involve lots of substeps that require some substantial development work including:

1. Connecting to various datasources
2. Writing queries to retrieve the data
3. Parsing the data into XML (or some other standardized format)
4. If interaction is needed, the data aggregator will also need to accept inputs and query dynamically

The issue is that all of these steps may differ among various datasources like oracle, csv, excel, etc. And yes, there are plenty of tools to help with this step. The main point though is that no matter what solution you choose, there will a substantial portion of development time needed for Step 1; even with a GUI you still need to write queries. It is really up to you if you want to follow this model and incur development time, however for an “executive” dashboard I feel it essential to follow this model.

The solutions I found can be divided into 2 categories: Web Reporting and Business Intelligence (BI). Web Reporting covers the basic aspects of the above model and nothing more; whereas the above model is just a subset of what  Business Intelligence can do. One thing to remember also is that its always “pay to play” with the support for the free solutions. They’ll give you the software, but you need to pay for their help. Also I tried to chose solutions whose free software should have all the features we need.

Web Reporting Solutions

• 

  Open Flash Charts Pie Chart

  
  1. Open Flash Chart – The simplest, (cheapest?) and fastest way to get some dashboards is to just write some code. If we are only connecting to one database, no problem using some free tools like Open Flash Chart. It is the simplest because we don’t really need to abstract any data, so step 1 processes are minimized. However it isn’t Enterprise level and makes big assumptions on the sources and format of the data. Any changes to the model will require code edits and support calls which doesn’t make for good sustainability.
• 

  Open Flash Charts - Area Chart

  
  2. WSO2 Web Application Server + Open Flash Chart – This combination is a little more enterprise. The tools are still free, which is great. The Web Application Server can handle step 1 pretty well and has a nice GUI interface to make all the data connections and output some XML as a web service, definitely enterprise level. However we still need a small middleware layer to accept the XML and generate the flash charts which require less (but still some) proprietary code as the first option. The only code changes that would be needed would be if you wanted say different charts, or a different layout, but could probably be done by someone with some basic HTML and scripting knowledge.
• 

  SAP Xcelsius Dashboard

  
  3. WSO2 Web Application Server + SAP Excelsius Engage – This combination is fully enterprise level. It could probably bet set up with no proprietary coding at all. Excelsius is stand alone program that allows one to graphically design the dashboards and connect them up with various datasources. The only problem is that it costs a minimum of $1000, and for the server edition (which I feel is unnecessary) could be as much as $5000. So that may be out of budget. But it does produce beautiful results.

Business Intelligence Solutions

I had never heard of Business Intelligence until I started doing research for this project. The best way I can describe it is that is a huge web based platform that seems to do everything. First off it has something called Data Integration. This is a very cool program that will not only aggregate different data sources, but allow you to transform among the datasources and output to pretty much anything. For example I could query an Oracle table and an Excel Sheet, aggregate those two into one dataset perform some transformation like sorting or pivoting and then output the result to a MySQL table. Its actually quite amazing. All of these data integrations can be set up as jobs that can be run in batch or run from command or run from some web service. From there it also includes a Web BI Platform, which is kinda like a portal that can show reports, dashboards, tables, basically anything and also has some admin for user control and permissions. It also includes tools to design and build reports and dashboards. It can also do complex analysis and data mining and will probably take out your trash and wash your car too.

Anyways, the benefits are intense, but may be a bit overkill if you just want a dashboard. Things this complicated can quickly turn from dream to nightmare however with these BI solutions almost all aspects can be managed through GUI tools. The real problem is that it will take a serious commitment to make it work because of its complexity. I am not sure they are the right solution for you, but maybe you will think of something else that it may be useful for.

• 

  Pentaho Dashboard

  
  1. Pentaho – A free, open source collection of BI tools. Again with the “pay to play” support. Pentaho offers all of the functions specified above and has GUI tools for managing all of them. I was able to actually install Pentaho and was able to generate some reports, however it was quite difficult to configure the server and generating my own dashboard required some very complex configuration. But the data integration tool was stellar, I think on its own it is a very powerful tool that could be used outside the scope of the BI platform.
• 

  IBM Websphere Dashboard

  
  2. IBM Websphere – I read all the papers for this but wasn’t actually able to try it out. But it is a huge system that probably does everything and I won’t even talk about it more because it probably costs many thousands. But it is a good basis to see what can be done.

Unless you see other potential value in implementing a BI system, I would say that option 3 under the web reporting is the best bet if you can spring for the software and development costs.

Here is the original Image I used. Initially I thought I could just use the basic image copy functions in GD and it would take care of the rest. Well this isn’t the case. This is the code I initially tried. Essentially creating a true color image and just using imagecopy to copy the gifs into the sprite.

 protected static function handleGif($spriteImages, $imgSize, $filePath){
    $sprite = imagecreatetruecolor($imgSize->right, $imgSize->bottom);

    foreach($spriteImages as $spriteImage){
      $tempImage = imagecreatefromgif($spriteImage->getPath());
      $position = $spriteImage->getPosition();
      imagecopy($sprite, $tempImage, $position->left, $position->top, 0, 0, $spriteImage->getWidth(), $spriteImage->getHeight());
      imagedestroy($tempImage);
    }

    ob_start();
    imagegif($sprite);
    $spriteOutput = ob_get_clean();
    imagedestroy($sprite);
    SpriteImageWriter::writeImageFile($spriteOutput, $filePath);
  }

Here is the output from the above code. You will notice that GD scrapped the transparency and opted for a black background. Back to the drawing board I guess. The next chunk of code shows the next iteration. Basically allocating the “black” color (which most gifs use as transparent) and setting the newly created image to use that as the transparent color.

protected static function handleGif($spriteImages, $imgSize, $filePath){
    $sprite = imagecreatetruecolor($imgSize->right, $imgSize->bottom);
    $trans_color = imagecolorallocatealpha($sprite, 0, 0, 0, 127);
    imagecolortransparent  ( $sprite, $transparent);

    foreach($spriteImages as $spriteImage){
      $tempImage = imagecreatefromgif($spriteImage->getPath());
      $position = $spriteImage->getPosition();
      imagecopy($sprite, $tempImage, $position->left, $position->top, 0, 0, $spriteImage->getWidth(), $spriteImage->getHeight());
      imagedestroy($tempImage);
    }

    ob_start();
    imagegif($sprite);
    $spriteOutput = ob_get_clean();
    imagedestroy($sprite);
    SpriteImageWriter::writeImageFile($spriteOutput, $filePath);
  }

And this is the output from the above code. As you can see we now have some transparency, however big chunks of the image have been made transparent. Specifically where the color in the image matched “black”, it was replaced by a transparent color. So last try now. Here is the final code, which first involves setting some flags that tell it to maintain the transparency. Then create a transparent rectangle over the whole image and next set the image color to transparent. So a little more overhead however the results are great.

protected static function handleGif($spriteImages, $imgSize, $filePath){
    $sprite = imagecreatetruecolor($imgSize->right, $imgSize->bottom);
    imagealphablending($sprite, false);
    imagesavealpha($sprite,true);
    $transparent = imagecolorallocatealpha($sprite, 255, 255, 255, 127);
    imagefilledrectangle($sprite, 0, 0, $imgSize->right, $imgSize->bottom, $transparent);
    imagecolortransparent  ( $sprite, $transparent);

    foreach($spriteImages as $spriteImage){
      $tempImage = imagecreatefromgif($spriteImage->getPath());
      $position = $spriteImage->getPosition();
      imagecopy($sprite, $tempImage, $position->left, $position->top, 0, 0, $spriteImage->getWidth(), $spriteImage->getHeight());
      imagedestroy($tempImage);
    }

    ob_start();
    imagegif($sprite);
    $spriteOutput = ob_get_clean();
    imagedestroy($sprite);
    SpriteImageWriter::writeImageFile($spriteOutput, $filePath);
  }

The above code will do the job of copying the transparent GIFs for my sprite generation. However one issue I ran into when generating my sprites is that when I had a lot of different GIFs in one sprite there was some small but noticeable dithering. I assume this is because while each individual GIF can use its own color pallete, when you combine a lot of them they can all only grab from the generated pallete.

In this post I will cover how to set up your MAMP and virtual hosts. I am using Mac OSX Leopard so some of this may be different for other versions. 

First after installing MAMP, Open MAMP and click on Preferences and then click on the Ports tab. Here click the “Set to Default Apache and MySQL Ports” button. This will change the ports and may require a password to complete.

Next lets download and install symfony. From now on when I refer to the project I will use myproject.

Let’s create a project directory, open up your terminal window and type this (change myproject to your project name)

$ mkdir /Applications/MAMP/htdocs/myproject

/Applications/MAMP/htdocs/myproject will be your web root from now on. Let’s now enter the terminal and checkout Symfony from SVN.

sudo /Applications/MAMP/bin/php5/bin/peardev channel-discover pear.symfony-project.com
sudo /Applications/MAMP/bin/php5/bin/peardev install symfony/symfony

Next in order to use Symfony on the command line let’s add a directory to the $PATH variable on the command line. First we need to check out path, so open up terminal and type this:

$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

So we need to take this path and append a ‘:/Applications/MAMP/bin/php5/bin’ on to it so in terminal it will look something like this, but if your path has a different starting configuration it may vary.

$ export PATH=”$PATH:/Applications/MAMP/bin/php5/bin”

Now you should be able to enter symfony commands in the terminal without the full path, you may need to restart the terminal for this to work.

Another way to do this is to just create a symbolic link to the symfony program in your /usr/bin like this:

cd /usr/local/bin
$ sudo ln -s /Applications/MAMP/bin/php5/bin/symfony symfony

So let’s initialize our symfony project

cd /Applications/MAMP/htdocs/myproject
symfony init-project myproject

You will want to create at least one application. From the terminal:

symfony init-app myproject

Next we need to open up the config/config.php in your myproject directory. And change it to something like this:

<?php
// symfony directories
$sf_symfony_lib_dir = '/Applications/MAMP/bin/php5/lib/php/symfony';
$sf_symfony_data_dir = '/Applications/MAMP/bin/php5/lib/php/data/symfony';
?>

Next let’s set up a virtual host with Apache. We need to open up MAMP’s httpd.conf file located at /Applications/MAMP/conf/apache/httpd.conf
We need to add our virtual host information at the very end of this file. Mine looks like this (remember to change myproject to your project name:

NameVirtualHost *

<VirtualHost *>
  ServerName myproject
  DocumentRoot "/Applications/MAMP/htdocs/myproject/web"
  DirectoryIndex index.php
  Alias /sf /Applications/MAMP/bin/php5/lib/php/data/symfony/web/sf
  <Directory "/Applications/MAMP/bin/php5/lib/php/data/symfony/web/sf">
  AllowOverride All
  Allow from All
  </Directory>
  <Directory "/Applications/MAMP/htdocs/myproject/web">
  AllowOverride All
  Allow from All
  </Directory>
</VirtualHost>

Now that that is done, we need to edit the /etc/hosts file. So open up terminal again and type:

sudo nano /etc/hosts
Then add 127.0.0.1  myproject to the end of the file. My hosts file looks like this:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting. Do not change this entry.
##
127.0.0.1       localhost
255.255.255.255 broadcasthost
::1             localhost
fe80::1%lo0     localhost
# VIRTUAL HOST START
127.0.0.1       myproject
# VIRTUAL HOST STOP

Next you will want to restart your apache. At this point you should be able to type in your browser

http://myproject

And see the default symfony page.

I usually like to run MySQL from the command line. So next I created a symbolic link to the mysql program into my /usr/bin like this in the terminal:

cd /usr/bin
sudo ln -s /Applications/MAMP/Library/bin/mysql mysql

Then I go into terminal to create my database for my symfony project (remember on MAMP default root password is root.

mysql -u root -p root
mysql >
create database myproject default character set utf8

And then in my project folder /config/databases.yml looks like this:

all:
propel:
class: sfPropelDatabase
param:
username: root
password: root
dsn: mysql://root@localhost/myproject

And that is it. Hope this helped.

I used a few of these articles to figure this all out:

• Installing Symfony in Mac with MAMP
• INSTALLING SYMFONY ON LEOPARD
• Installing Symfony on Mediatemple