Search Engine Optimization (SEO)

This is a topic that I had to research recently for a class I am taking in graduate school. Search Engine Optimization (SEO) is a technique for increasing the traffic to a web site from search engines. (e.g., Google, Yahoo!) In a nutshell, SEO involves tagging, designing web site content using keywords and metadata (information about data), and designing web site structure for efficient accessibility by search engines algorithms and web crawlers to find information more effectively.

The desired goal of SEO is to attain a higher ranking for a specific web site that results in the site showing up in the first page or initial few pages where most users tend to click on a resulting link before moving onto another search or simply abandoning the result set. This in effect makes the web site appear to be more relevant to the user in the search result listing. The reason why search result set relevance and position is critical is because of the observed behavior where most web users tend to use the first page of a search result listing most of the time. Subsequent pages in the search result listing are used by a much lower percent of web users as the search engines become more efficient due to SEO techniques and continual improvements to search engine algorithms.

Search engine optimization means ensuring that your Web pages or site is accessible to search engines and are designed in ways that help improve the chances they will be found. ("Intro to SEO")

Some decent simple SEO Examples are described here. There are many techniques that make your web site "SEO Friendly". One simple technique is to choose a SEO Friendly Domain name. Let' say you have a web site that contains SEO information. If your site is located at mysite.com, it is not a very good "SEO friendly" domain since it is difficult for a search engine to figure out that your site contains SEO information.

On the other hand, if you named your site search-engine-optimization-info.com it would be considered more "SEO friendly" by its name alone. Using this domain name any search engine can figure out from the domain name alone that your site probably has SEO related content. (e.g., cars.com, salesforce.com)

According to Google, SEO is an abbreviation for "search engine optimizer." Many SEOs and other agencies and consultants provide useful services for website owners, from writing copy to giving advice on site architecture and helping to find relevant directories to which a site can be submitted. What is an SEO? Google's definition. Google does not have any relationships iwth SEO organizations and does not make recommendations. However, they do provide tips on choosing an SEO and information on what will make a search engine drop your site from their search results entirely

Additionally, Google provides a wealth of information about its search results, improving site rankings, and tips that help Google's crawlers find your web information more efficiently and effectively. With Google representing the majority of web search today, approximately 67% as of March 2008 and growing, it is wise to know how to optimize your web sites for Google web crawlers.

I also found some really decent material on SEO techniques here.

The Unexpected for 2008

As anticipated, Google released the unexpected a few weeks ago. It is called Google Sites. If you need to create a web portal for sharing information amongst a team, classmates, workmates, social mates, friend, or family, then Google Sites may be for you. Here is the overview of Sites:

"Google Sites is the easiest way to make information accessible to people who need quick, up-to-date access. People can work together on a Site to add file attachments, information from other Google applications (like Google Docs, Google Calendar, YouTube and Picasa), and new free-form content. Creating a site together is as easy as editing a document, and you always control who has access, whether it's just yourself, your team, or your whole organization. You can even publish Sites to the world. The Google Sites web application is accessible from any internet connected computer."

I have had a chance to play with Sites and it will brings "online everything" and cloud computing even closer to reality today. This is another disruptive event initiated by Google and will give everyone something to think about when it comes to collaborative sites and portals. If they can make this work with Gears and be taken off-line, then I think IBM, Microsoft, and Java based portals will have to re-think their respective strategic agendas.

caGrid and caBIG uses open source Java technology

While working on a research project for graduate school, I serendipitously stumbled upon the cancer Biomedical Informatics Grid (caBIG). This is a massive bioinformatics initiative to connect the entire cancer community leveraging modern information technology (IT) to save precious time toward news discoveries in cancer research. The caBIG Essentials tutorial is a must. It is available as a an online interactive tutorial or presentation download.

caBIG uses open source technology to bring cancer research into the 21st century. It is intertwined into a grid middleware infrastructure, caGrid, which is an information grid for biomedical research. (Bioinformatics Journal article abot caGrid) The really interesting aspect of caBIG and caGrid is that it uses open source Java technology and UML to make this all work. It is a service oriented architecture (SOA) using Apache Tomcat, Axis, UML, BPEL, Hibernate, etc. providing open specifications to allow organizations to adopt or adapt its IT systems to be caBIG Compatible.

The caGrid concept initiative originated December 2003. (caGrid wiki with timeline graphic) The initial prototype was brought online in July 2004. The following two and half years of work resulting in the caGrid 1.0 officially going live in December 2006. caGrid 1.1 was released in September 2007. This cutting edge work is in progress and will change the way biomedical informatics research is peformed in the future.

caBIG is model driven. That means you must have object models UML in order to transform your IT resources into compatible caBIG applications and services. caBIG provides/maintains the specifications, tookits, and SDKs. The possibilities for biomedical research breakthroughs leveraging the emerging bioinformatics field are endless.

Javascript 2.0 goes OO

This is good news for the future. The emerging Javascript 2.0 specification brings object-orientation to Javascript. See this article, Web 2.0 Meet Javascript 2.0.

This is a welcome advancement in the continuing evolution of Web 2.0 capabilities and bringing rich "online everything" to reality.

Non-existent Object Oriented User Interfaces in 2008

The state of object-oriented user-interface (OOUI) technology today is quite depressing. I bet that you can not name one that is useful and in production. After all the years since I stopped using OS/2 and its innovative WorkPlace Shell (WPS) OOUI, there is yet to be anything like it in Linux, Mac, Unix or any other mainstream operating system. The closest we have today is the Mac OS X which is really an intuitive graphical user-interface (GUI).

The heart of the IBM OS/2's WPS is the System Object Model or SOM. SOM is an object-oriented system level object model that is similar to Microsoft' Common Object Model (COM). The key difference is that SOM supports inheritance while COM does not. So, in esseence, COM is really an API with object-oriented like features. SOM is truly object-oriented.

Back to the OOUI discussion. WPS objects had the capability to inherit from parent classes or other base classes. One clear feature of WPS that I miss in the modern GUI/OS world of today are Shadows. There currently is not similar analogy to this in KDE/GNOME, Mac OS X, or Windows.

In WPS, Shadows objects that point to other objects. They are like shortcuts in Windows except that they inherited all their properties from SOM. The one thing that is still a major annoyance with shortcuts is when you move the file or directory to which your shortcuts is pointing. The shortcut is really a static pointer to some file, directory or application. When you do this you get the invalid shortcut problem in Windows. The same scenario occurs in KDE/GNOME and Mac OS X. You would figure that this problem would be resolved by now. But that is not the case. Instead there are tools, utilities, tips, workarounds, scripts or whatever to fix this very trivial issue.

In an OOUI like WPS, Shadows inherit from SOM the File, Directory, or Application object. So when the underlying File, Directory, or Application changes, your Shadow automatically knows this through inheritance. This is very simple, intuitive, and just works.

I recently read that the motions by the OS/2 community (yeah, it still exists) to request that IBM open source OS/2 has been denied by IBM due to several licensing and intellectual property reasons. OS/2 source was developed collaboratively by IBM and Microsoft. On the other hand, SOM is an IBM technology that was developed after IBM and Microsoft split in the early 1990s. I just read a new suggestion ("Should IBM' SOM/DSOM be open sourced?") that IBM explore open sourcing SOM which would give to the world an object model that is proven and can be a foundation for future OOUIs developed using a modern open source methodology.

A move by IBM that makes SOM available would inject a stimulus into the state of user-interface innovation which appears to have stagnated in the past 10-years. In general when you look at KDE/GNOME, Mac OS X, Windows, and any of the variants used in Unix, they all pretty much do the same thing. They are some form of implementation of the WIMP (window-icon-menu-pointer) paradigm that Xerox PARC created in the 1970s.

I think it is time that we focus on the OOUI again. It seems to have lost its lustre so far this decade. Maybe then things will get exciting in the UI space again. If a way to tie the OOUI to the web and make it some form of OOUI that is browser based, an object-oriented web interface (OOWI) if you will, that would allow it to work on any platform! Now that would be real progress. Then we can start talking about OOWIs! Oooh weee! Wishful thinking.

Is This Disruptive Thinking?

The just announced Microsoft unsolicited offer to acquire Yahoo! for $44.6 billion is all over the news. Analysis of it cover the spectrum. One question I have is, "Is this disruptive thinking on Microsoft's part?". One thing is certain, the fear that Microsoft has of Google appears to be reaching a paranoia state. This is almost an admission from Microsoft that they did not get the software as a service (SaaS) right. SaaS appears to be outside of its 'core competencies' even though they have invested billions in the Internet.

This take over offer appears to be an admission by Microsoft that they have failed in their efforts to transform itself from a desktop Windows applications centric company to a software as a service company that leverages the web. This evolution has been an ongoing objective for the past 7-years with nothing but a 44% drop in stock value.

Google's revenues and influence on the web keep climbing while Microsoft and Yahoo keep declining. The combination of Microsoft and Yahoo will only yield about 20-30% of the web search market. Google currently owns about 60-65% of that market and keeps increasing its share.

Microsoft's achiles heel appears to be its goal to tie everything (software applications) to Windows. Yahoo and Google on the other hand are internet service based companies whose goals are to make all their services available to anyone, anywhere, using any operating system or web browser. "Is Microsft Saving Yahoo! Or is Yahoo! the Savior".

I think that a merged Microsoft and Yahoo! will encounter several cultural issues. They have vastly different cultures. The cultural intertia of both companies will be difficult to grasp and refocus into a convergence. It will be a daunting task.

There is a significant technology overlap that would cause a lot of internal disruption and team dynamics issues that are not evident on the surface. Look at how the TimeWarner-AOL deal turned out. These two distinct cultures clashed and resulted in the decline of AOL. Orchestrating a successful merger of Microsoft and Yahoo! will take some extremely well calculated and organic management decisions in a timely manner.

Google is not sitting still. It appears that the Microsoft merger's strategic goals are addressing the services for which Google is already successful. Knowing Google, they are moving on to the next thing whatever that is. We will find out sometime this year in the typical Google fashion, they will just drop it onto the world.

Basically, Microsoft is playing follow the leader. If Microsoft has failed so far in competing with Google, the merger will only complicate things for them because of all the internal reorganization that is required to make it successful. Overcoming the structural interia in Microsoft and Yahoo! is a complex strategic management undertaking. These internal tasks will be a distraction for the new combined Microsoft-Yahoo corporation. This distraction if not resolved quickly will give Google more time to innovate and pull further ahead. "Fitting the Pieces Together in an Internt Mega-Merger".

One thing is certain, 2008 is already full of suprises and it is only February!

Happy New Year 2008

Happy New Year 2008!. Ok, it's early in the 2008 and it is always interesting to attempt to forecast out what may occur this year. Last year, I predicted that Google would do something unexpected and they did. Android was something entirely unexpected. Google Presentations was inevitable.

So what does 2008 bring? I think we will see Blu Ray become the HD standard as HD-DVD starts fading away. The reason for this is the Sony PS3. Additionally, just last week Warner Brothers announced that they will no longer support HD-DVD.

Apple Mac OS X will attain a 10-15% or greater market share of personal computers and start making headway in the business sector. This is based on the trend that has been 2007 for Apple.

Linux will become more significant as new devices like the Asus Eee PC, Nokia 810, and alternative ultra portable internet devices based on Linux start being used as the replacement of notebook computers. The mobility, power, inexpensive portable storage, solid state disks (SSDs), ubiqutous WiFi, and rich web applications are making this possible. These new devices have been disruptive in nature and are rapidly becoming the preferred devices.

Mozilla Firefox has attained a 35% market share globally according to Browser Statistics as of Nov 2007. The increase in Firefox usage will continue throughout 2008 and move in the 40% range at a minimum.

New phones using Google Android will redefine smartphones and the ultra portable internet devices market. This will be disruptive to the status quo wireless industry. It will be nice to be able to develop in Java for the Android based devices. What will be nicer is eventually a Firefox browser running on Android. If the services are right, 2008 will be the time for me to get into the smart phone market as user. Up to this point, the Apple iPhone is a step in the right direction. Based on all this, I think Microsoft's Windows Mobile will be the loser.

As they have done in the last few years, I think Google will do something unexpected again in 2008. Apple will refine its products and continue to make cool stuff that everyone wants. Sony will finally get its online 3-d immersive virtual reality business established and leverage the PSP and PS3. Microsoft's market share will continually take hits from all fronts. Java will continue to get larger because of Android, Apple, PS3, Blu-Ray, and just because there has been so much innovation and revival within the Java community in the past year.

Other than that, 2008 will be yet another exciting year of information technology innovation!

Disruptive Technology in 2007

2007 is the year of disruption. At least in my opinion. We are in the final month of the year and looking back I see several disruptive technologies that will change the technology landscape in the next few years. If you look at the following list which is not in any significant order other than alphabetical you will see what I mean

Android
Asus Eee PC
Google Presentations
gOS
Grails
Groovy
Knol
iPhone
iPod Touch
Mac OS X Leopard
Nokia N810
OLPC
OpenJDK
OpenSocial

Each one of the above technologies is creating a change wave that affects an existing market space or is the catalyst for an emerging market. These technologies are actually causing competition to re-evaluate its current market and live with the threat of being displaced.

Naked Objects Pattern

I am sitting here early on a Saturday morning at a local Honda dealer waiting for my vehicle to be serviced. While online and drinking a good cup of java (coffee), I had not written to this blog in a while (been quite busy) so decided to discuss a new Java technology that I have been researching lately which is quite impressive and ground breaking in my opinion.

While researching Java technology, I encountered a new pattern to which I was previous unexposed. It is called the Naked Objects pattern. What was surprising to me is just how little coverage this design or architectural pattern receives in the popular cyberspace. What started me down this path was the What is the Matter with JMatter? article I read last week. After reading that article and doing quite a bit of exprimentation with JMatter, it was the Naked Object architectural pattern that is most interesting.

I will not go into too much detail about JMatter, but in a nutshell it implements the Naked Objects architectural pattern and throws in some domain-driven design concepts to provide a very capable Java application engineering framework. The result is an extremely agile development environment for Java technologists. At first I thought JMatter was for the average user. After experimenting with it for a week, I have come to the viewpoint that JMatter is for the exprienced Java developer that understands design patterns, Ant, Hibernate, Spring, dependency-injection, XML, Swing and a few other best of breed Java frameworks that it leverages. The result is a several order of magnitude increase in application development productivity. What would have taken weeks or months to build can now be engineered in days or weeks.

The JMatter applications are currently client/server and leverage Java Web Start for deployment. What is on the drawing board is better usage of WingS and Echo2 to generate AJAX capable applications based on the JMatter application. I tested this capability that is integrated into JMatter today and it requires a bit more maturity before I would recommend using it. For now, if you an experienced Java developer, JMatter is worth a look. It changed my entire perspective on Java application development which I now call application engineering. Using JMatter and its capabilities allows you to engineer applications vice developing them in the traditional sense, lots of coding.

Google Sky, Wow!

I predicted Google would do something unpredictable and it occurred earlier this week. They released Google Sky which is integrated into the already incredible Google Earth. If you are into astronomy or just want to see what type of imagery is available of space, then this is it. Google has taken the collective imagery from the science community (NASA, etc..) and put into one place that anyone can now use and navigate just as you can with Google Earth.

I used a few astronomy programs and Google Sky definitely is the best one I have seen to date. I can't wait until there is a similar add-on for the Moon and Mars. Currently, Google Moon and Google Mars are accessible as 2-D browser applications.

Web OSs and the Future

I like looking toward the future and making bold predictions. Here is one, the WebOS paradigm will take over within the 3-5 years. Check out the list of existing WebOS projects at Wikipedia.

I have experimented with a few of these and just recently installed eyeOS which is a really impressive WebOS concept. It is written in PHP and is extremely lightweight. I read a really good review of eyeOS and then tried out it. Another decent concept is YouOS which is written entirely Javascript. It is currently in a proof of concept phase but has promising future if it is continually supported and matures. I have used the Sun Java Desktop System which is is very similar in concept however does not execute entirely within your web browser.

As the WebOS paradigm matures, I think it will eventually take over what virtualization and other desktop paradigms do today purely because a WebOS lends itself to execute in an extremely lightweight container, the web browser, and lends itself to exist in embedded web browser objects executing in mobile and new generation of personal digital devices (i.e. iPhones, GPS devices, PS3, Wii, DS, PDAs, smart phones, PAVs, etc.)

The Firefox Kid and a new Web OS Project

While drinking my first cup of coffee on this wonderful Saturday morning in July, I bumped upon an article at Slashdot about Blake Ross, The Firefox Kid, and his latest project, Parakey. The details about Parakey are sparse because it it still in development. However, what is interesting is the peek that Blake has allowed to date.

In a nutshell from what I have gathered, Parakey is an open-source based application that instals on your desktop and runs in your browser and bridges the gap between desktops and the web. In essence, it is like a Web OS. As the project matures, we will see more details.

You have to read the article to get the full scope. There is also some interesting historical information about how Blake got involved in the Firefox creation.

Here is an interesting note, Facebook just purchased Parakey for an undisclosed amount. The we will continue be a highly dynamic place.

Firefox Has 28% Marketshare in Europe

I predicted this a year ago. At the rate that Firefox usage has been increasing it appears that Firefox now consists of 28% of European users. "Europe Helps Life Firefox Market Share". In some countries usage is as high as 48%! Whoa! At any rate, I think Firefox's ascent will continue especially when Firefox 3.0 comes out with its capability to support offline web applications similar to Google Gears technology.

If you are interested in web analytics, here are a few sites that clearly show the Firefox trend.
W3Schools Browser Market Share
Market Share Hit Links

What will be interesting is how this trend continues through 2007.

Offline Web Applications with Google Gears

As I expected, the unexpected innovation has been released by Google. I actually started using Google Gears the day it was announced from a user perspective when they enabled it in Google Reader. It definitely works and is actually a significant breakthrough at the tip of the ice berg.

Over at OnLAMP they put out some articles using Google Gears with PHP and MySQL. The Power Of Google Gears (part1, part2). It has only been a little while so let's see how this offline browser technology evolves. Mozilla is working on a similar technology in the next release of Firefox. Maybe we will see more of this later this year.

Grasshopper 2.0, Wow!

While perusing a few websites last week I enountered an article that piqued my interest at Internetnews.com, "Mainsoft Brings ASP.NET 2.0 to Linux". What the article really covered was the Mainsoft Grasshopper 2.0 product announced 6/6/2007 which lets you build your ASP.NET 2.0/C# or VB web application and targett for deployment on a Java/J2EE servlet container or portal.

I have been spending the past month working with the .NET 2.0/3.0 framework and have had the opportunity to test Grasshopper 2.0 using VS2005. The early verdict is, 'wow'! This is truly an accomplishment. The Mainsoft team has created a Visual Studio 2005 extension that converts Microsoft Intermediate Language (MSIL) bytecode to Java bytecode from a C# or VB ASP.NET 2.0 web application. It actually works as advertised. Grasshopper 2.0 fully supports .NET 2.0. The resulting product is a fully compliant Java .WAR that deploys to Tomcat-5.5 as you would expect.

Mainsoft provides a free Developer Version of Grasshopper 2.0 which I used. It requires an activiation key you must get from their website. Installation into VS2005 was flawless as long as you have VS2005 with SP1 installed. The Mainsoft tutorials are easy to follow. If you know how to build an ASP.NET 2.0 web app using VS2005 then using Grasshopper 2.0 is seamless. You just use your existing C# or VB skills in VS2005. You do not need to know Java in order to use it. That is intent of the product.

The Mainsoft online documentation, FAQs and tutorials are all decent for experienced developers. In addition to the Developer version there is the portal and enterprise versions which add additional features. The Mainsoft website provides a clear explanation of each version's features.

Now what would really be cool is Mainsoft Grasshopper supporting SharpDevelop. SharpDevelop is the best open source IDE for .NET that I have seen.

A Google Summer

Google surprises again in this first week after Memorial Day. If you have not been paying attention or maybe are getting into the nice weather we have been having towards the end of May, Google has updates Google Maps with Street Views. Nothing really new since Microsoft has had this in Live for about a year now. The difference is that the Google Street Views actually works nicely, is intuitive, and does it all in all browsers.

I toured San Francisco and in the new Google Maps Street Views and I am totally impressed! It is really an immersive 3-D experience. Wow is all I can say.

And then there is Google Gears! This one is just as impressive. It is an offline web application framework that works in all web browsers and platforms since is based on Javascript and a local SQLite database. I installed it on my MacBook and on Windows XP in Firefox and had no issues at all. Google has made Google Reader a Gears enabled application and it works just as advertised offline. Cool stuff.

Now, they just need to get GMail, Docs and Spreadsheets working in a similar transparent fashion. If Google Reader is any indication of what the near future brings, then 2007 will turn out to be quite an innovative year for Google. The real killer is that Google Gears is released as open-source under a BSD style license.

I think Microsoft is going to be in trouble and losing ground on the web front. I plan to see how Google Gears works out for enabling custom web applications to enable offline browsing capabilities. Time will tell how successful Google Gears becomes.

Microsoft vs. FOSS

Well, I expected this was coming any day now and that day has arrived. Microsoft has initiated a new front against Free and Open Source Software (FOSS). The CNNMoney article is a good read, "Microsoft Takes On the Free World", about the front Microsoft intends to open concerning specific patents it says Linux and other open source software violates.

The next week and the future will be interesting. The article discusses a 'cold war' like scenario in software technology putting Microsoft and the FOSS in direct legal collision course. FOSS has powerful corporate patrons and allies. In 2005, six of them - IBM, Sony, Philips, Novell, Red Hat and NEC - set up the Open Invention Network to acquire a portfolio of patents that might pose problems for companies like Microsoft, which are known to pose a patent threat to Linux. There is speculation that the intent of this is to provide a 'mutually assured' litigation scenario similar to the cold war 'mutually assured destruction' scenario.

FOSS and others are on the opposing front with their own arsenal of patents ready for its use as a counter attack against Microsoft. Looks like the SCO law suit all over again but this time a more frontal assault. IBM won the SCO lawsuit. SCO was indirectly supported by Microsoft funds. An interesting fact to note is that IBM holds the most patents in the software industry and has not threatened anyone that I know about with its patents.

What is interesting is how the CNNMoney article states that half of the Fortune 500 use FOSS (Linux) and growing. The article alludes to Google which is widely known to be a large proponent of open source. One area that appears to be the real threat are online free web applications in which Google is moving towards very rapidly. Microsoft must really be hurting or at least is significantly threatened by Linux. Their fear of Google is well known.

It could be that the future trends with the migration towards open source and Linux will undermine future Microsoft business. I have read many blogs and analysis to this effect ("Microsoft is Dead", "Eight Signs Microsoft is Dead in t he Water" -John Dvorak) and the most recent actions by Microsoft appear to be supporting this theory. What is ironic is that Google never confirms or denies that it is competing with Microsoft. Microsoft appears to be obsessed with Google's success.

On yet another front, it was just recently reported that Apple's sales for MacBook accounted for near 10% of all notebooks in March 2007. Apple is also #4 on the notebook sales list. It looks like Microsoft is trying to open up too many fronts and maybe needs to review history and the art of war.

The next few years will be very interesting.

MVC Frameworks Update

I have had the chance to take a look at the recently released Struts2 framework. It was made generally available in late February 2007. It is based on the WebWork2 framework which in itself has been in active development the last few years.

It appears that the Struts2 team has provided a nice update to Struts which has been showing its age and actually has lost a lot of its advantages due to advances in the Java world and the dynamic scripting language (i.e. Ruby, Perl, PHP). Here is a decent Struts2 history worth a read.

Some of the nice features of Struts2 are simplicity, plugins, Spring integration, AJAX support, and flexibility to use POJOs for form and action classes. The requirement for Action and ActionForm mappings no longer exist. It integrates with Freemarker, Velocity (no more VelocityStruts), and many other frameworks.

If you have been looking at MVC frameworks in Java lately, I think Struts2 is worth a look. If you have works with Struts in the past, the learning curve is minimal. It will interesting to see how much traction Struts2 gets this year in the Java web applications arena. There are many competing MVC frameworks and Struts is still widely used. I have been leaning towards simplicity lately and it looks like the Struts team has been spending the last year doing the same.

It is good to see that Ted Husted is involved. This adds a lot of confidence and credibility to the Struts2 project. In a addition to Struts2, I have been looking at Spring MVC and the Spring WebFlow frameworks. They do not appear to be getting as much attention this spring (no pun intended). Anyway, it's good to see some action in the Struts community. Since I have not been following the Struts development in the past year, it was a nice surprise to finally see a general availability release.

I guess I will be spending the next few months with Struts2 and see what it can do.

Digital Woes for Mars Global Surveyor

It appears that a single command to the Mars Global Surveyor was the root cause of it eventually failing. "Mars Global Surveyor Dies From Single Bad Command". If this is truly the case then this is a pretty scary event considering that a software engineer somewhere is responsible for the loss of a remote space craft.

I read a book a few years ago called "Digital Woes" written by a Reagan era 'Star Wars' computer scientist, Lauren Ruth Weiner, that cast a serious doubt on the probability of succsss of such a complex system. The probabilty of success based on the statistics of digital failures in the modern computer age was calculated to be too high. From what I recall, there was a lot of statistical and scientific models created where the success of the 'Star Wars' system was unachievable.

At any rate, the news today concerning the Mars Global Surveyor reminded me of the book and how software engineering is even more critical today. A single bad command caused a cascading set of failures which eventually shutdown the space craft. This just adds fuel to the argument that software testing is critical and that sound software engineering practices may not be enough.

Based on what I know about what happened to the Mars Global Surveyor, it looks like the NASA team followed all disciplined processes but the failure still occurred. It appears that the complexity of the system behavior was not well tested which resulted in the sequence of events leading to the eventual failure of the system. The LA Times article mentions budget cuts, political pressure and short time frames as contributing causes for the human error. The Scientific American article also mentions human error.

I wonder if they did any modeling and simulation or at least simulation of the scenario? Being an optimist, I think that test-driven software development is not enough with the complexity of modern systems and that simulation is required in order to really observe the behavior of the system. Given the declining NASA budget environment and fewer resources, maybe some new ways of approaching software engineering at NASA is an option. The 'do more with less' mentality requires new ways of thinking and a lot of engineering management innovation.

Remember OS/2, It is 20-Years Old Today

I just saw this OS/2 20th birthday posting. I was an early adopter of OS/2 in the 1.0 days. Then used it well in into the late 1990s. Those were good old days. OS/2 is still alive and kicking.

Wonder when IBM is going to open source it? Now that would open quite a few opportunities! We can still hope and wish.

Virtualization on Mac OS X

Check out my blog entry I just posted over at The Mac Katana. It discusses my experiences on Mac OS X with VMware Fusion Beta and Parallels.

Online All The Time

In my lifetime of communicating with a computer, I have lived and been a part of several paradigms of online communcations. I remember the dial-up the BBS days in the 1980s, the CompuServe forums in early 1990s, e-mail in various forms of evolution in the 1980s through the 1990s and the spam infested world of today, groupware in the 1990s, and instant messaging, forums, blogs, portals and wikis in the past few years.

Collaborative online communication continues to evolve. Today, the social networking sites, personal wikis, personal blogs, and portals are all evolutionary forms of storing and managing online information. What I like about today's world is that with open source technology and the numerous online services providing you with all the tools you can imagine, the only thing you need is a modern web browser to access it all! Not having a blog, participating in a forum, being part of some online community, or not being online is like not having access to a telephone in the 1970s.

I remember being a young person in the late 1970s and early 1980s and spending hours talking on the phone. For me, this has translated into the modern equivalent of being online all the time today. With notebook computers, internet gadgets and all types of mobile devices, it is hard not be online today. You actually have to make a effort to stay offline today. The only time I spend less time online is when I am on a vacation. Then I intentionally disconnect.

I find that instead of watching TV and cable, I actually do this while staying online. A few years ago this was sort of a novelty. Today, it is now just a normal way that I watch TV/cable and concurrently browse the internet. My multitasking capabilities are up and my attention span is going down. I am not sure if this is a good thing but I have adapted to this way of personal interactive entertainment over the last few years.

What I have not seen is the smooth convergence of TV and the web that was envisioned in the 1990s. This did not occur. Today, we still have TV/cable that pretty much works the same way it did in the 1990s and the web has become richer but the convergence has not occurred. I think the two industries are intentionally trying to figure this out and no one really wants to the change it all. YouTube and other online video sites are in a big fight with traditional content providers and from what I have seen, it appears that the traditional content providers are resisting the convergence and change. So what else is new?

Well, back to my original topic. I am watching a Comedy Central show right now and writing this blog entry. Maybe in 10-years we'll see the convergence of TV and the web. I doubt it because that is what everyone thought 10-years ago in 1997.

Search In 2007

Today, everyone knows how to use Google or a search capability. It is very intuitive and even my son who is in first grade has figured out how to use the search feature in online dictionaries like Merriam-Webster. What surprises me recenlty is the how the enterprise search market is becoming a very competitive space.

A few years ago, Google and Yahoo! were the two enterprise level products on the market with an low cost to entry. This is a critical point in the modern IT environment. Today, there are numerous open source and proprietary solutions all claiming some type of advantage in their implementation of search technology. I can name quite a few search technologies today that I have reviewed Google Search Appliance, Yahoo SDK, Amazon A9 OpenSearch, IBM OmniFind, Corpora Find!, Microsoft Sharepoint 2007 Enterprise Search, Nutch, Lucene, Autonomy/Verity, and whole lot of embedded search engines. In the Java world, Lucene appears to be the best of breed search framework. As a matter of fact it is the basis for the IBM OmniFind Yahoo! Edition search technology.

I really like the Google appliance mentality. You just install it and use it. It just works kind of like your refrigerator. When was the last time you had to do any type of maintenance, patches, reconfiguration, optimization, of your refrigerator. You do not! That is the point. You just use it, it works and you keep it clean and it looks good. That in summation is what I consider the baseline needs and wants for a search solution. No one else has anything like it on the market yet.

I have designed and built solutions using Lucene, Verity, Google API and Yahoo! SDK. Generally speaking, they all work in a similar fashion. Personally, I prefer the appliance approach that Google provides. It just works. It uses the same Google API that you can already use and implements a dedicated hardware based solution that requires very little systems administration overhead.

As for the state of enterprise search today, I would say the the IBM OmniFind Yahoo! Edition is a powerful open source entry into this highly competitive space. With it's low-cost to entry and open source foundation, it looks like it will mature into a viable competitor to the Google Search Appliance in the next few years. Comparing the Google API to the Yahoo! REST based search SDK, I think the Yahoo approach using REST is much easier to use from a developer perspective.

I would not count out the open source product Nutch either. It is based on Apache Lucene which is the defacto best of breed Java indexing framework. Nutch has been brought into the Apache Software Foundation which is a powerful force in the open source universe.

As for the state of search, I think the leaders remain the same and the fresh new open source competition is breathing life into this area that was once completely dominated by a few products.

Platform Agnosticism and Ultra-Portable Computing Machines

I have been on the front lines of the platform wars for more than a decade now. I was an early adopter of 32-bit DOS extender technologies (DesqView, Concurrent), OS/2, 32-bit Windows, Linux, Citrix, virtualization and other platform specific extension technologies for which some have since long come and gone. Today, virtualization, cross-platform languages, Java, fast processors, cheap memory, cheap storage and open source technology have changed everything. The concept that a specific platform will dominate no longer fits the paradigm shift of the always connected online future.

I have spent the past five years working towards platform agnosticity and have only recently been able to experience it in practical form. It has been three months since I switched to using Mac OS X as my preferred environment. Since, then I have been able to truly practice, research and experience cross-platform computing and software engineering. Java, being designed cross-platform is a key enabling technology.

The online Google Docs and Spreadsheets and all their online applications has pushed the envelope even further. I now use my online Google apps more than anything else and use my desktop applications (OpenOffice.org, MS-Office) as online extenders for necessary functionally where ever I am. This is an inversion of how most people perceive these online applications. Anyway, I have lived this inverted environment since last summer (almost 8-months now) and can confidently say that it works for me and is the future of computing. I blogged about this last year. ("Online Everything Is Getting Closer") This year I think the "online everything" is even better because I have been able to live it and push the innovation further. Google's recent announcement of Google Apps for Enterpises is a testament. There is also a rumor that an online presentation tool is in the works! That would be impressive.

I have found that open standards, open source technology have had a larger influence in the mutli-platform world more than enything else. Prior to switching to Mac OS X, I was primarily a Linux and Windows user/developer using Java technology. I always preferred Linux overall but now my MacBook has become a 'better Linux than Linux' in my experience. With Linux there was always something missing in the GUI (KDE, Gnome) application spaces. On the Mac this missing gap is closed by the numerous robust GUI applications that exist on the Mac OS X platform. In addtion to the excellent Java support built-in to Mac OS X, there is an active user base/community that just focuses on the getting things done. As Apple has been pushing, 'It just works.'

This year is shaping up as I anticipated. I have moved most of my computing work online and can access and collaborate online using any computer platform, anytime, and anywhere. My search now is for ultra-mobile computing device that has a full-blown Linux like OS allowing me to access all my information whenever and where ever I need. This new device must not sacrifice any computing power or external peripheral capability via USB or Bluetooth interfaces

I recently checked out the Sony UX series devices which are incredibly tiny machines based on Windows XP Professional. It uses solid-state hard drives (no moving parts) which is highly desireable in an ultra-portable device. Nokia has the N770 and N800 Linux based devices which are getting closer. I have not seen the N800 in person yet. I have been using my Sony PSP as a portable browser for two-years now and it just works as browser in addition to being an excellent multimedia gaming device. I also recently got to play with the Sony Mylo hands on and this is getting closer but not quite there yet. Some people like the Mylo but for me, it does not meet the full-blown computer requirement.

I am hoping that Apple comes out with something in this ultra-portable device space that 'just works' like my MacBook does. I tried out my mobile phone a few years ago as an internet platform and very quickly ruled that out because it is 'too small' and limiting as a computing device for power users or developers. Not much has changed in this space recently. Plus the bandwidth is not there in the US for the mobile smart phones. Today's mobile phones are too small and too slow for practical use as a serious computing platform.

I am looking for a 'better Linux than Linux' in a ultra-portable form factor that uses solid-state storage with no sacrifices. My best bet desirable device today would be a Linux based device that is ultra-portable. Maybe Apple has a ultra-portable Mac OS X device in the works? I like the Sony UX series but being a Windows device and relatively expensive puts a negative spin on it. Java is a key open source enabling technology that must be supported given its scalability and cross-platform design and having it is my preference being a Java technologist. Given the state of wide Java support today in Linux, Mac OS X, Unix, and Windows I do not see this as an issue.

Anyway, the migration of moving everything online allows me to be very critical of the ultra-portable computing device I want in the future. Notice that in the ultra-portable environment, I really have not focused on any platform. Just the device form-factor, features and capabilities as an ultra-portable computing machine that sacrifices nothing. No one has attained it yet but I know what the target requirements are for my ultra-portable computing machine are and it does not yet exist. I prefer something that is Linux/Unix centric since that is where most of the innovative ideas and cutting edge computer science research occurs today. Maybe later this year someone will surprise the world and come up with an innovative new product! I think in the next few years we will definitely see this happen. Whoever produces just has to remember that cost is also a factor.

Yahoo Pipes

Leave it up to the IT industry to come out with a new product out of the blue. Yahoo Pipes has the opportunity to be one of the killer apps of 2007. I was just experimenting with RSS aggregation in java and php last year. I had a lot of good ideas but never go around to them due to the usual not enough time to focus on any of the ideas. Recently, I found a decent java webapp called Flock that is no longer maintained but has the right idea. Flock could use some nice enhancement. And then just yesterday, Yahoo! announces Pipes!

Details of Pipes are hard to confirm since the site is overloaded and they have a nice banner message stating that "Our Pipes Are Clogged" due to overwhelming demand. From what I have read, it appears to be some type of RSS aggregation/mashup tool that provides some visualizations on what you are creating. If it is anything like the Yahoo! Site Builder, then it will probably be a rich client java application that provides a nice GUI.

At any rate, when I finally get a chance to see Pipes first hand in the next few days of weeks will blog about it and see if it stands up to the over hype.

Nice Java Development Platform

I have been using my MacBook for almost two months now and feel quite comfortable with it. I have been using it exclusively at home working and experimenting with Java web applications that are running on my Linux servers. All I can say is that if you are looking for the next thing and are not tied to Windows then Mac OS X 10.4 Tiger is it based on my experience.

The Mac OS X 10.4 Tiger user-interface is very intuitive and the Finder, Dashboard features are exactly what I was looking for in a next generation GUI. OpenOffice.org, Eclipse, and all my other tools for Java development work as advertised on Mac OS X. The next release Mac OS X 10.5 (Leopard) has even more features that take it to the next level.

For Java development, I can now say that Mac OS X is a really good solution. Much better than Windows primarily because of the underlying Unix foundations. If you are administering Linux servers and need an extremely consistent GUI, the Mac OS X is it. Your mileage may vary as I always like to say.

I created a new blog, The Mac Katana, devoted to my experiences on the Mac. Check it out if are interested.

I wish everyone good luck happiness and joy in this new year, 2007!

Project Looking Glass 1.0 3-D UI

The future of user interfaces will be 3-D and it looks like the Sun has been busy with its Project Looking Glass 1.0 3-D user interface. "Sun Releases 1.0 Looking Glass Interface". As for the future, the 3-D user-interfaces are qutie impressive from the screenshots I have seen. You have got to check them out for yourself.

Just noting it here since Sun recently put a 1.0 out there for the open source community to experiment with and evolve.

OpenOffice.org 2.1 on the Mac OS X

I have been using OpenOffice.org on Windows and Linux for many years now. I just recently got myself a MacBook and have spent the last three weeks using it and learning how Mac OS X 10.4 (Tiger) works. I have to say that I am getting more comfortable using the Mac OS X. I am using Firefox as my browser, have gotten used to the various Mac OS X applications and am now getting serious about using my new Mac as a software development machine. I primarily develop in Java so getting all my development tools setup on the Mac is a must. This includes Eclipse, NetBeans, Tomcat, JBoss, Nvu and OpenOffice at a minimum.

Well I must say that OpenOffice runs fine on the Mac OS X after finally getting it installed last week. Currently you must have X11 running in order to use OpenOffice on the Mac. A pure Aqua version of OpenOffice is in beta and should be ready for primetime in 2007. Look here for details. I downloaded NeoOffice which runs fine however, for me I noticed a few performance issues with NeoOffice. So I reverted back to OpenOffice as my word processing, presentation, spreadsheet, and desktop database suite of productivity applications.

What really amazes me is how brainwashed so many users are with the perception that they 'must have' Microsoft office in order to function on a computer. I met several Mac users that said they purchased Office for the Mac in order to do word processing. Wow, the marketing machine at Microsoft sure does have everyone fooled.

This is the reason why I am blogging about OpenOffice on the Mac. It is available today and works as advertised. If only more Mac users or recent converts were aware of OpenOffice and its capabilities they could save themselves a few dollars and rid themselves of Microsoft. Personally, I enjoy have a Microsoft free laptop using my new Macbook. I have had the same on my Linux notebook for years and it sure does feel great not having to be dependent on Microsoft for my personal computing requirements.

Too bad that I work in a predominately Windows environment at work. It would be nice to see Macs brought into the workplace. Based on my experience, it just works like my refrigerator. No strange behavior, bugs, or just plain oddness to worry about anymore.

My Initial Mac

After years of working with Windows, Linux, OS/2, I have always wanted to get into the Mac but never had the motivation or reason to spend the money to get a Mac. Well with the release of the Intel based Macs earlier this year and the ability to run Linux and Windows in virtualization via Parallels and soon VMware I took the plunge and bought me a MacBook on 11/29/2006. After seeing the features of Windows Vista and waiting for its release, I don't see any compelling reason to upgrade to Vista within the year or so unless it occurs at work. For personal home, entertainment, and personal multiplatform research use, I am going with Mac OS X.

After using it for a few days I am happy to say that I really like the Mac OS X 10.4 Tiger and the Mac in general. There are some extremely intuitive and unique capabilities on the Mac OS X that I have not seen on Linux, OS/2, or Windows. The Mac OS X 10.4 Tiger Dashboard, Expose, HotCorners UI features name just a few. These are the features that I have been using extensively. There is this UI automation feature which I haven't even gotten into yet. I forgot the name of it but it looks exactly the type of thing I was looking for in a 21st century operating system.

As far as multimedia is concerned, the Mac just blows away Windows in my opinion and experience. I have used Windows Media Center Edition and compared to the Mac, it looks like a half-baked experiment. The Mac multimedia device experience is like an extremely polished and intuitive appliance. It is just something you have to experience personally is all that I can say.

I still have a lot to learn about the Mac architecture, Darwin, Cocoa, Aqua, X11 on the Mac and more. After just running on it for a few days, I am excited to say that I like it! I will be blogging a lot more about the Mac in this blog and more in the next few years. Given that I do almost everything on the web or online now, switching to a Mac was a piece of cake for me. Mozilla Firefox and Safari work nicely and I had almost nothing to learn from a user perspective.

From a software engineering perspective, Mac OS X totally embraces Java which is good for me since I have been working with Java technology for a long time. I have been finding quite a bit of open source stuff for the Mac which is nice to know. I really like all the Dashboard Widgets. For the future, I have been looking at the previews of Mac OS X 10.5 Leopard that Apple has been putting on their website and it has some innovations that again will make my next Mac OS X upgrade in 2007 a more capable Mac. The new Spaces and TimeMachine capabilities are a step into the future.

I will continue working with Linux and Windows since that is how I make a living but it would be nice to be able to make a living working on Mac OS X for web development. At any rate, I hope your future computing is as enjoyable as I foresee mine will be. After just a few days of using my MacBook, I feel better about my computing future already. In addition to all the above, I forgot to mention that my MacBook just aesthetically looks and physically feels cool under my fingers.

Playstation 3

Today, 11/17/2006, Sony unleashed it's Playstation 3 (PS3) to the US market. Ok, so what does the Sony Playstation 3 have to do with Software Engineering? Quite a bit if you look out into the digital future, at least in my opionon. The PS3 is much more than a game machine. It is a supercomputer based on the IBM/Sony/Toshiba designed Cell processor (I blogged about the Cell earlier this year, The Octopiler) that runs on Linux, has a 60GB hard drive, 802.11g WiFi, Gigabit Ethernet, Blu Ray disc, and is designed to be the center of a digital entertainment hub for all your multimedia desires. Oh, yeah it is also an incredible Hi-Definition (up to 1080p), networked gaming machine.

I will not bore you with the technical details because that can be found all over the internet, printed media, and whenever you need it. One very interesting aspect of the Playstaion 3 that is downplayed, at least initially, is its inter-connectivity with the PSP. I really like the portability aspect of the PSP and use it quite frequently as a portable web browsing device and gaming machine. I'd like to see the PS3 and PSP collaborate with one another in games and other forms of digital entertainment that have not been explored yet.

I don't expect to be able to get a Playstation 3 anytime soon due to sell outs and limited supplies initally, but when I do eventually get one in the near future I intend to set it up in my family room and give it a spin as a center piece for the digital living room of the 21st century. I have intentionally held back from getting a large HDTV until after the PS3 was released. Now that the PS3 is out, I have begun researching and shopping for an LCD HDTV that will provide me with 1080p capabilities.

I think the PS3 will make the digital life interesting and fun again in 2007 and beyond. Something I would like to see is a Cell based notebook computer running Linux! Now that would be a really cool evolution say in 2008 for this family of technology. For now in 2006 and in 2007, we have the PS3. I can't wait to get mine.

The Open Source Battle Front

November 2006 will be remembered as the month when proprietary software companies opened a battle front in the inevitable war between open source and proprietary technologies. Here are some major signifcant events that have occurred:

• Oracle announced that it will compete with Red Hat Linux directly in the Linux IT services sector.


• Microsoft announced and signed a deal with Novell for licensing and working with the Linux community.


• Microsoft also has signed a deal with Zend the open source PHP vendor.


• Sun made Java open source under the GPL on 11/14/2006.

These are just the major announcements. There are many more than this however, the big proprietary companies are significant since they command so much market share and industry influence.

So what is the agenda under the agenda? It is an acknowledgement and action by the big proprietary vendors that they have decided to compete directly with open source companies that threaten their proprietary software business model. I thought this batte front was going to happen in 2007/2008 and it caught me off guard as to how soon the chosen battle fronts were made by the proprietary software companies (i.e. Oracle, Microsoft).

At any rate, the next few months and 2007 will definitely be a year of adventure, changes, and interesting developments in the open source world. What is interesting is how Google, Yahoo, Amazon, IBM, and other pro-open source companies have stayed out of the fireworks so far this month. It looks like they are all managing to proceed with their open source strategies, whatever they are, and moving forward changing the world.

As a supporter of open source I think big changes are in store for all us in 2007 and beyond.

Unstructured Information Management

Unstructured information makes up most of the information content on the internet today. Estimates are as high as 90% of available information on the internet is unstructured. So with all the databases, portals, websites, repositories, hard drives and trillions of files that exist today how do you harness this information? This is where the field of Information Management has the technical challenge to turn all this information into useful information and knowledge. This is entirely conceiveable given sufficient time, computing power and storage. The challenge is making this happen in near realtime.

To meet this challenge, DARPA has funded IBM Research in 2005 to create UIMA which stands for the Unstructured Information Management Architecture. It is an open, industrial-strength, scaleable and extensible platform for creating, integrating and deploying unstructured information management solutions from combinations of semantic analysis and search components. IBM makes UIMA available as a free SDK (alpha), and makes the core Java framework available as open source software (UIMA at SourceForge) to provide a common foundation for industry and academia to collaborate and accelerate the world-wide development of technologies critical for discovering the vital knowledge present in the fastest growing sources of information today. IBM developerWorks has a tutorial for using the UIMA SDK with Eclipse.

Since IBM released UIMA as open source in early 2006, it has been widely adopted. Open source projects such as GATE, OntoText, and many other have been utilizing UIMA as the framework for unstructured information management research. As research into managing and harnessing unstructured information grows, there will be more available solutions to solve these problems.

The Expanding Google Earth

If you have not used Google Earth lately then you are missing out on one of the killer applications that merges the web with rich native applications. The 3-D Warehouse of Google Earth plugins has been growing steadily after Google released Sketchup, the 3-D modeling aplication. Sketchup comes in a free version and a professional version. The nice thing about this approach is that you get to use a fully functioning program that scales to the professional level if you need that capability.

"The growing world of Google Earth" provides some insight into just how Google Earth has been evolving. I use Google Earth quite frequently as the means to find places and to explore places that I have never been. You can spend hours doing this and is quite entertaining.

The real power of Google Earth is the API provided by Google that allows you to build upon Google Earth's capabilities by integrating it into your custom applications. Additionally an SDK is provided to allow even more customization. Google Earth is becoming a platform for creating new applications or mash ups.

The imagery database is constantly being updated and I can see the day when the 3-D models for just about everything is integrated and available. I can think of several applications of integrating Google Earth technology and designing 3-D programs for everyday use. For now, we all get to watch this killer application evolve slowly which is ok with me.

Open Source To Rescue OCR

This blog is not entirely about open source intentionally. It just happens that open source technology is where all the action and innovation appears to be happening in software today. I just bumped upon an article about OCR that interested me. "Google releases open source OCR tool". If you thought at OCR was a stagnant technology then you just joined the crowd and Google's viewpoint.

What is interesting is the fact that it took an open source mentality, a stagnant technology (OCR), and a few bright Google engineers to reinvigorate the OCR technology space. The new open source product called, Tesseract, is released under an Apache Software License. Google hopes that the open source community will create solutions and new products for the OCR technology.

Prior to reading this article, I was not aware that OCR technology has literally stagnated for the last 10-years. HP and UNLV released their OCR technology into open source to re-generate interest. Well, it appears that Google does have the interest. Remember the digital library initiative that caused an uproar. Well, open source OCR looks like it is par for the course of working towards that goal.

The Next Set of Open Source Waves

I have written about my viewpoint of open source technology and how it is transforming the IT industry. Well, I am glad that this mentality or at least thought process is becoming mainstream. I really like the angle that InfoWorld has put on this in the article "Who are the Losers, Now That Open Source is Winning?".

What is even more ironic is how Microsoft has approached the Mozilla foundation to collaborate on future projects. This is just more evidence that open source is a strategic aspect of any company working in the IT industry. You can no longer ignore open source if you want to remain viable in the future.

The success of Google, Yahoo, Amazon, EBay, SalesForce.com, RedHat, JBoss... are case studies for businesses built on open source technology. Getting back to who is winning and the next set of waves, the InfoWorld article above makes a pretty good assessment of the current state of open source vs. proprietary technology in the IT field. The bottom line is for the next few years there will be peaceful coexistence. After that the two will be on a head on collision course and if you are not a player with or in open source technology, you will probably not be doing business successfully in the future.

As always in this business, time will tell and we are always just another major breakthrough away from a complete disruptive technology shift. So far this year, I think the cards are playing in favor of open source and its increasing critical importance to the IT industry.

LAMP Stack Development

Well, I have recently been pretty heavy into PHP development and must say that is quite an change compared to Java/Struts and J2EE type development. One nice thing I have found is taht Eclipse and Zend are working together to make this a smooth process with the PHPIDE. If you are a Java developer and already use Eclipse, you will feel quite at home using the Zend PHPIDE which is an Eclipse 3.2 plugin that works quite nicely even in beta form.

There are pros and cons to using the LAMP (Linux, Apache MySQL, Perl/PHP) or WAMP (Windows, Apache, MySQL, Perl/PHP) stack. I will just refer to this as the LAMP stack from now on in this posting. One benefit is that all the tools are open source. There are very few financial barriers to using these technologies. The primary con I see if that you have to ensure that you are using compatible versions of each of the components in your technology architecture. One solution is just try a VMware appliance, WAMP, or XAMMP packages pre-built for Linux or Windows. The downside to the pre-packages approach is that if you have to deploy and support a LAMP/WAMP stack, you will not have the necessary configuration skills for troubleshooting issues that may occur on your servers. I recommend that if you need to support this stuff, then learn to do it the hard way at least once so you will know the configuration and mechanics of the LAMP stack. The "Install & Configure Apache, PHP, JSP, Ruby on Rails, MySQL, PHPMyAdmin & WordPress on Windows XP/2000" and "Building a LAMP Server" HOWTOs really helped me out with this stuff among other HOWTOs I found. As usual, the HOWTO article was not bullet-proof or 100% accurate. I think I found about a 75% accuracy rate for configuring LAMP. Most issues I have found concern the PHP5 and MySQL5 issues (see this Zend post).

In other words, I discovered that MySQL 5 and PHP 5 have introduced some connectivity issues and incompatibilities with latest Apache HTTP Server. PHP5 doesn't even work with Apache 2.2 in my experience and I confirmed after doing quite a bit of research on the web. From what I have seen, the PHP community is working on supporting Apache 2.2. For MySQL5 and PHP5 you need to be using Apache HTTP Server 2.0. I learned this the hard way. Additionally, I was not aware that MySQL5 breaks quite a few PHP applications due to changes in password encoding and introduction of object-orientation in PHP5. Then there is the PHP mysqli extension that supercededs the mysql extension that is built into PHP. What a frustrating experience that is.

Anyway, what I have learned with all this experimentation with LAMP stack is that it is like all the other open source technology solutions I have been working with in the past few years. It does work with a few caveats. LAMP works as good as your understanding, skills and ability to use the web for research all the tools and components. Paraphrasing this, LAMP works if you are willing to put in the time and have the right mentality and necessary open source skills to solve problems.

Open Source Technology and National Security

I have read several articles that support the idea that the US DoD should adopt open source and open technologies more readily to match the productivity gains that many companies in the private sector have attained. The recent article, "Open Source in the national interest", provides a strong case for adopting open source in the DoD more readily. The 79 page April 2006 report mentions specifically how IBM has transformed itself by adopting open source readily.

There was a 2002 Report by Mitre that advocated the same. The 2002 article is not in as much depth as the more recent article noted above.

If Google, Yahoo, EBay, Amazon, IBM and many others are doing so well with open source technology on the large scale, then I think our government can do the same. It would just require a transformation of the government IT organizations. This is definitely more difficult than it sounds but when I see entire countries adopting open source technology (i.e. EU, Japan, Korea, China, India) I get that feeling that we might be falling behind or losing our edge by not following suit. I think it is in our national interest and so do many others in this ever changing computer science field.

Agile Web Development and the Lighter Languages

I have spent the past few months working with developing and experimenting with Java (JSP, Struts, Spring, Hibernate), Perl, PHP, ASP, Visual Basic and Ruby. Well, I will have to say there is something to be said about agile processes and open source dynamic languages. Recently I have had the opportunity to work with Java/JSP, Perl, and PHP. I am porting a PHP web application to JSP and intentionally sticking with a JSP Model 1 implementation. This is a bit of a digression in development techniques or a few steps backwards but I am testing out a theory. I can always refactor to an MVC design or other if necessary.

The theory that I am testing is: "It is possible to ignore elegance temporarily for attaining speed and practicality of getting a solution working using a 'quick and clean' process. (agility)" The old cliche in programming of doing something 'quick and dirty' to just get the job done is a concept I never really liked but have had to do occassionally in my career. The 'quick and dirty' approach almost always got me in trouble. I try to avoid doing this at all costs if possible. Based on my experiences with agile 'quick and clean' development, I feel that it is possible to attain software development speed practically in a sustained manner without sacrificing too much elegance. In other words, getting the cleanest job done in the best possible manner in the shortest period of time.

What I have found interesting in the days I've spent porting PHP to JSP is that it is relativley straight forward if you ignore your inner programming voice that says you should refactor the PHP to a more elegant Java/JSP design. In just a few days work, I have functioning webapps in JSP that rival the development speed of the looser PHP. I have to note that the PHP webapp is not using any PHP framework, design patterns (Five Common PHP Design Patterns), or other MVC pattern frameworks. This would of course complicate the design and make porting PHP to Java/JSP slower in my opinion but would require an experiment to prove it.

One other thing I must note is that having a really good IDE is critical to making this work. I am currently using Eclipse 3.1, Netbeans 5, and Zend Studio 5 in these 'quick and clean' experimental processes and this is definitely a must. I recently added Netbeans 5 to my pallete of java programming tools and it has yielded quite a productivity improvement for JSP and servlet development for me.

My next set of experiments is to get into Ruby on Rails (RoR) development which is a 'quick and clean' architecture. I have been talking about this all year however, have not made the time to start an experiment with RoR.

Archaeology and FOSS Trends

Archaeology has been a passion of mine since I was a kid. Prior to getting into computer science and music, archaeology was one of the professions I wanted to pursue. To this day, using the internet and reading as much as I can I try to stay involved in what is current in the archaeological field.

Much of the software technology used in the archeological field has been proprietary until very recently. I just bumped into an open source product called ArcheOS that is a GNU Linux distributionhas specifically configured for archaology! The 1.0 release was presented late in 2005 and has since been evolving following the open source model.

In the article, "Uncovering progress in FOSS-based archeology", it discusses the progress made to date with FOSS for the archaology. I will research more into what ArcheOS does and should have a cursory working knowledge of this stuff sometime this year if I can devote a few days to it.

I am surprised at how FOSS has been finding its way into the archeological field. I guess with continuing technology trends, open source technology appears to be invading just about every aspect of life where software is required and used. It makes logical sense that open source technology is finding its way into other fields other than business, science and technology.

Windows Genuine Advantage Spyware

I need to stand on top of my Microsoft soap box and say a few things about Windows security. I use both Linux, Windows and whatever devices make my digital life more productive and entertaining. Well, if you have not heard, just this past week Microsoft was caught and tagged as the largest spyware source to date. "Microsoft Big Brother". For roughly the past year, all modern Windows PCs have been phoning home every day. This was put into service in July 2005 as the Windows Genuine Advantage (WGA) service which was required to allow Windows patches to continue working. The true intent of WGA was never revealed by our trusted Big Brother, Microsoft.

The thing that bothers me the most about this is that Microsoft did this for security reasons and used it for some yet unknown reason (marketing) to make using Windows better for everyone. This sounds like a smokescreen to me and makes me wonder about the new integrate security features like anti-spyware Microsoft has embedded into the next version of Windows Vista.

Boy am I glad that I have options and know how to use Linux and other personal computing platforms effectively. As I look to the future, I see a less Windows dependent future for me. At least as far as home computing and entertainment computing and security are concerned. With Linux, Mac and Sony PS3 coming online soon, I think in the near future I will further diversify my personal computing technology architecture and use Windows less. Choices exist, you just have to exercise your right to choose based on good solid information.

Online Everything Is Getting Closer

I moved to using e-mail on the web in the mid 1990s with Yahoo! Mail. It was the best thing since slice bread for me. Prior to using Yahoo! Mail exclusively for my personal email, I was always on the wrong computer or at the wrong place when I wanted to access my e-mail folders containing the e-mail with information I needed where I was. This scenario could be while at home needing to access a work e-mail or while at work and having to access a home e-mail. Keeping track of the e-mail accounts and which computers could access it was just a pain.

To make a long productivity story, once I moved 100% to web e-mail, the problem just disappeared and is now a no-brainer. I do not understand the number of people I know who are still tied to thick client e-mail (MS-Outlook) today. Why? Who knows.

Well if you are still using your thick client calendar or e-mail, then you are stuck in the 1990s. With Yahoo! Calendar and most recently Google Calendar and the ease of collaboration over the web only requiring that you have access to the internet using any browser, I just don't understand the folks I hear talk about Outlook and wishing they were at home to access their personal email while away from home. Huh!?! Hello?! It's the 21st century and that is an old 1990s problem. Why are you still stuck in the 1990s?

Well, with the recent release of Google Spreadsheets and Google's acquisition of Writely, I think the days of being tied to a thick client for spreadsheets and word-processing are numbered. The only pieces missing is a nice web-based presentation application and a quick webtop database for business productivity. I suspect this will happen within the next year.

I can think of a few other web apps that would be nice replacement for their thick client equivalents, however for now I'd like to reserve some room for innovation in 2007. By the way, I just got my Google Spreadsheet account today and have already been inspired with a few ideas for its use within the first few minutes of using it. As for the "Online Everything" concept, I think this was dubbed Web 2.0 last year.

Google Web Toolkit makes AJAX Easier In Java

Today Google released the Google Web Toolkit (GWT) which is a Java framework that makes AJAX web applications easier to develop. I got my first taste of GWT today and have to say that Google has an AJAX product that makes building Java AJAX applications much easier to do. At least compared to my experiences in the past year.

A few of the features are that you don't have to write any Javascript. In comparison to other Java-based AJAX framework like DWR, you had to write some Javascript code. GWT generates all the necessary Javascript code for you! Google provides good examples for you to learn GWT.

Based on my first impression of GWT, they also provide decent documentation of GWT online. Anyway, this was a surprise that I happened to see in "Google Releases AJAX Framework".

The Modern Configuration Management System Solution

The de facto standard for source code version control is Concurrent Versions System or CVS as it is better known. Well, CVS has been around for a while and has a few flaws that most organizations just overlook. I used to work for one of those organizations that applied the "if it ain't broke, don't fix it" mentality toward source code version control. I have used PVCS, CVS, Teamsource, a few others and most recently Subversion.

CVS was good enough and it did what we required, most of the time. There were a issues with how CVS handles file renames, directory name changes, etc. which we just lived with like everyone else using CVS.

Well if you happen to use CVS today and just deal with all its binary file handling, file name change, directory handling flaws, I have some news for you. You should really consider migrating to the modern successor to CVS, Subversion. I had the opportunity to research, evaluate, test Subversion 1.3.1. I really should have done this a few years ago but now is as good a time any.

Subversion was designed to be a better CVS than CVS and I based on my experience they attained this goal. The primary reason for this is that I installed, tested and evaluated Subversion on Windows servers and it works as advertised. CVS is best installed and run on a Linux server. Subversion runs well in Linux and Windows.

Now I see why many of the Apache Software Foundation projects have switched to Subversion from CVS. Subversion handles binary files with ease and has all the nice GUI tools and Eclipse plugins that CVS has. I installed tested and really like RapidSVN, SmartSVN and TortoiseSVN. The Subclipse plugin for Eclipse 3.1 works very nicely. I did not find a good Netbeans 5.0 plugin which was my only disapppointment. Other than that, Subversion 1.3.1 is definitely ready for prime time. The Subversion website is an excellent source of information and tools available for Subversion. There are CVS to Subversion conversion tools which should make your transition smooth. I have not tried any of these repository conversion tools myself so I caveat my previous statement by saying your mileage may vary.

I found that the information in "Subversion UI Shootout" article is still relevant today. In the article "Setting up a Secure Subversion Server" which is geared towards Linux, the concepts can be applied to a Windows server deployment. For J2EE/Java EE environments, I found JavaSVN described in "Configuration Management in Java EE Applications Using Subversion" to be an excellent framework for Java server side development if you happen to need a servlet interface with your subversion repository.

For configuration management solutions today, I will be migrating everything I do related to configuration management to Subversion. It is definitely a better CVS than CVS and in my opinion, a good candidate for a modern open source configuration management system for more than just a source code version control system. I recommend that you use it for all your configuration management requirements if your organization is not adverse to open source technology. Good luck.

Google's Latest Products In April 2006

Google has been busy in their software engineering division. They recently released SketchUp which is a 3D Modeling tool. "Google Launches Free 3D Modeling Software". SketchUp allows you to create 3D models and store in Google Earth.

Google released Google Calendar in April 2006 and in the usual Google style it is dubbed a 'Beta'. Well, if you have not had a chance to try it you should. I have used many web-based calendar products like Yahoo Calendar, Lotus Domino, Mozilla Calendar, Sunbird, and even Microsoft Outlook Web Access. Well, I will have to say that Google has done a good job of entering an already crowded market and managed to make a dent and establish some traction.

Like their other recent products, Google has integrated many AJAX features which makes this product really useful. I am most impressed with the ease of use and responsiveness of Google Calendar compared to the competition. For my first set of impressions, it is as responsive as Google Maps. Which I suspect is not an easy feat. The software engineers at Google have been quite busy.

The ease of use is the most important aspect of the new Google product. Scheduling events and appointpoints is the easiest I've seen in any web calendar or desktop calendar. You just click on the date and an AJAX style window pops up allowing you enter your event. This surprised me. The ease of sharing your calendar and keeping a private calendar all in one interface is innovative. This feature is much easier to use compared to others I have seen.

The intuitiveness of Google Calendar is the most impressive. To reschedule an evevent or appointment you just drag and drop. Wow! All this from within my web browser.

For now, Google Calendar has become my favorite web-based calendar software. The integrate AJAX features and the responsiveness of the product are what impress me the most. There is still some refinements to be made like integration with hand-held devices and mobile phones. I am sure these features will eventually find their way into Google Calendar. This will be an interesting product to watch evolve.

Enterprise Portals

Portals, SOA and web services are the new buzz this spring. Well if you are involved in software engineering you are probably involved in projects that deal with content management, search, portals, SOA and some form of web services to expose all this content. So what is a portal? For the context of this posting I am using this definition of a portal. For open source java technology I try to stay focusing on JSR-168 compliant technology.

I have had the opportunity to spend some time researching portals lately and have found quite a few decent open source tools that are quite mature. eXo is a enterprise level product used by many organizations. The eXo platform is a powerful JSR 168 compliant enterprise portal built from several modules. It's based on Java Server Faces, Pico Container, JbossMX and AspectJ.

One product I like at least based on initial research is Liferay. It leverages all the best Java frameworks (Struts, Spring, Hibernate, Velocity, WSRP, MyFaces, etc..) and provides a really mature open source portal product. It has been in development for 6-years and is currently in version 4.0 release.

Over at the Apache Portals site, they are brewing Pluto which is the JSR-168 reference implementation, Jetspeed-1, Jetspeed-2, WSRP-4j and Graffito. All the Apache products are actively managed and are evolving fast.

For additional java open source portal technology, the following index contains a listing of quite a few more portal products. On the non-java front, Plone looks promising. It is built with Python. There are quite a few more open source portal products out there which are all evolving.

On the commerical side, there is Documentum which is a suite of enterprise level web applications. Documentum is large and deep. It is another JSR-168 compliant product. I mentioned it in a previous blog posting, "SOA, Data Warehouses and Modeling". Vignette is another product which I am still researching. Lotus Domino is even converging with WebSphere and becoming a player in the portal market.

So which enterprise portal products are the best? With the complexity of portal technologies that is a very difficult question to answer. I been involved with the debate that Microsoft Sharepoint is the solution. Based on my experience as a user of Sharepoint, I think it is somewhat limiting and rudimentary as a portal compared to the technologies I have seen. Everyone seems to be evolving in the direction of some type of enterprise portal. I like to stay focused on JSR-168 since it seems to have the momentum in enterprise portal technology today.

A Universal OS and new RAD Web Tool

Well, it looks like the operating system wars are coming to an end. Metaphorically speaking, this is the equivalent of the destruction of the Berlin Wall. It looks like due to the overwhelming economics of fighting open source Linux, Microsoft has decided to embrace the open source model, restructure, and support Linux with all its energy. You could see this coming with the recent reorganization announcements within Microsoft's management ranks. Recall that this type of a adaptation worked for Apple when it made a similar decision in the late 1990s for its Mac OS X project.

Economically, sustaining the battle against the open source movement was a draining the corporate giant. With the EU on the verge of fining Microsoft $2.4 million a day for violation of EU antitrust rulings, this has proven to be too much business risk to Microsoft's bottom line. Microsoft did not have any intention of complying with any of the EU's rulings which is exactly what they have done in the US in the past 10-years. Lobbying the EU just does not appear to be working as it has in the US.

In addition to the Singularity project that Microsoft R&D is working on, they have decided to join the Eclipse foundation, support the Novell Mono project, and have announced that internally they have been working on a Mac OS X like project where the Windows kernel will now be based on the Linux 2.6 kernel. The operating systems engineering group has been working on this for the past 3 years and they are nearing a point where they can unveil their progress to date to the world in true Microsoft marketing style, PowerPoint presentations. We will start getting the marketing briefings shortly as Microsoft prepares to adapt. There is a rumor that this project was initiated to counter the growing competition between Microsoft and Sony.

Sony has been collaborating with Apple, IBM, and Toshiba engineering a Linux based operating system that makes heavy use of virtualization and can seamlessly run Linux, Mac, Windows applications with ease. It is designed for the Cell chip and we should be seeing alpha versions of this new yet to be named OS in early 2007. I even saw that it will run on the Playstation 3 which makes a lot of sense.

Now you can see why a 'Universal OS' is on the horizon or at least there is OS convergence happening on all fronts. All the major players are working on some form of Linux kernel based operating system for the future.

On the software development front, the Eclipse foundation and Mono have announced a secret project that will permit developers to build Java, .NET, php, Ruby web applications using a universal web GUI/AJAX RAD plugin. Based on rumored information about early alpha versions, it's capabilities can be summed up to be equivalent of what the initial Borland Delphi 1.0 did for GUI client/server development in 1995. It changed everything as far as RAD client/server development is concerned. As a matter of fact, I saw some early hints that the IBM was purchasing Borland's IDE tools (Delphi, JBuilder) which are now on the market primarily to keep the talented Borland software engineers employed within the Eclipse Foundation. Things that make you go hmmm.

I have been waiting for a 'RAD for the web' type of tool that will give developers back the productivity levels that were attained with Delphi/Visual Basic in the 1990s. In the past few years, open source tools have come to dominate the web development space outside of .NET. With the infancy of Web 2.0, it looks like tools consolidation and a new dynamics have the prevailing winds blowing in the direction of the open source model. I personally have seen many decisions made for this movement allowing organizations to adopt open source technology where Microsoft once ruled. Look at what open source technology adoption has done for the emerging commercial space industry, "The Software of Space Exploration".

2006 is shaping itself to be a year of innovation, excitment, and surprises just as I anticipated. It is only the first of April and yet so much change is happening! I anticipate the next 9-months of this year being just as exciting.