BI feeds

Is the post positive and/or balanced?

YES NO Click here to go to the start of the decision tree

You have discovered an article, blog, comment about your organization, hobby, domain or interest. What is the date on the post, is the post still relevant?

YES NO Click here to go to the start of the decision tree

You can provide a factual and well-cited response, which may agree or disagree with the post, yet is not factually erroneous, a rant or rage, bashing or negative in nature. You can concur with the item, let it stand, or provide a positive review.

Do you wish to respond?

YES NO Click here to go to the start of the decision tree

Is this a site dedicated to bashing and degrading others?

YES NO Click here to go to the start of the decision tree

Is the posting a rant, rage, joke, or satirical in nature?

YES NO Click here to go to the start of the decision tree

Are there erroneous facts in the posting?

YES NO Click here to go to the start of the decision tree

Is the posting a result of a negative experience?

YES NO Click here to go to the start of the decision tree

Avoid responding to specific posts. Monitor the site for relevant information and comments. Notify your organization if applicable.

Click here to go to the start of the decision tree

Do you wish to response with factual information directly on the comment board?

YES NO Click here to go to the start of the decision tree

Packt Publishing has released their book on PDI – Pentaho 3.2 Data Integration : Beginner’s Guide

The book was written by María Carina Roldán, a valued member of the PDI community. She does a great job of fulfilling the promise of the book’s title – it is a great introduction and ‘getting started’ book for Pentaho Data Integration. I have worked with PDI for the last few years, so I’m hardly a beginner, but I still learned useful techniques and tips from the book.

The book starts with basic information about the tool and installation instructions, then, chapter by chapter, it introduces features and techniques that build upon each other. By the end of the book you’ll know how to perform all the standard ETL operations using Pentaho Data Integration and then some. There are tutorials and exercises throughout the book that really you understand the tool and its features.

If your organization uses Informatica, Ab Initio, IBM/Ascential Data Stage, Business Objects Data Integrator, or Cognos Decision Stream, this $45 book could lead to savings of hundreds of thousands of dollars with Pentaho Data Integration, and make you a hero in the process. If your organization can’t afford those products, this $45 book could lead to solutions at a fraction of their cost, and make you a hero in the process.

Almost everything in the book applies just as well to PDI V4.0 as it does to V3.2

Disclosure: I was one of the reviewers of the book during its writing so I might be a little biased – but I was not paid for that effort, nor am I being paid to review the finished book.

The U.S. Air Force have guide for Airmen on how to, and not to, use social media. It’s an interesting document. Their guidelines are generally applicable for most organizations. I particularly like their ‘Web Posting Response Assessment’ (last page of the PDF) which is a flow chart for how, and if, to respond to a article, post, or comment online.

http://fcw.com/articles/2010/05/05/air-force-social-media-guide.aspx

The main guidance is on pages 10-15, and page 19 – the sections titled: ‘Blogging’, ‘Industry Practices’, ‘Guidelines for Popular Sites’, and ‘Top 10 Tips for Social Media’

The PDF URL is:

http://www.af.mil/shared/media/document/AFD-091210-043.pdf

Since the flow chart is Public Domain I’m including it:

Dear Kettle fans,

A few weeks ago, when I was stuck in the US after the MySQL User Conference, a new book was published by Packt Publishing.

That all by itself is something that is not too remarkable.  However, this time it’s a book about my brainchild Kettle. That makes this book very special to me. The full title is Pentaho 3.2 Data Integration : Beginner’s Guide (Amazon, Packt).  The title all by itself explains the purpose of this book: give the reader a quick-start when it comes to Pentaho Data Integration (Kettle).

The author María Carina Roldán (blog, twitter) is a seasoned BI consultant and a valued member of the Kettle community. Besides her frequent appearances on our forum, she is appreciated by many for the time she spent on the Kettle Tutorial.

I’m not going to go over the detailed table of content.  Since I wrote the foreword of the book, I’m sure you’ll agree I’m somewhat biased. However, in all objectivity, the book covers what it claims to cover: it does help the PDI/Kettle beginner tremendously.  It covers all you need to get started and then some: the installation of PDI, the typical “Hello World” setup of PDI, reading text files, calculating, scripting, databases, repositories, etc.  As the title indicates, this book covers the current 3.2 stable release of Kettle, not the upcoming 4.0 release. However, for as far as 99% of the topics covered are concerned, that shouldn’t make too much of a difference.

So obviously I can recommend this book very much. It’s a time-saver for those that are starting with PDI.  For those that have dabbled with Kettle before I must say that María packed the book with nice tips and tricks so I’m sure you’ll be able to learn a thing or two.

Until next time,

Matt

Microsoft has signed the Joomla Contribution Agreement: http://community.joomla.org/blogs/leadership/1167-microsoft-signs-jca.html

Matthew Aslett at the 451 Group posted about it here – Tilting at Windows or (don’t be a Cnut)

I agree with him.

Reading the comments on the announcement is interesting. I see two kinds:
* Paranoid, irrational, subjective, and discriminatory ones objecting to Microsoft’s participation
* Well-balance, pragmatic, objective, and open ones cautiously welcoming the news

The basic issue is the same one as with the Oracle/MySQL situation, which I wrote about at the time: http://jamesdixon.wordpress.com/2009/05/14/open-source-fanatics-choose-wisely/

As with Oracle/MySQL there are calls to preemptively fork Joomla to prevent Microsoft participating. This is sad, self-limiting, counter-productive behavior. Worse it is directly counter to Joomla’s OSI license. To quote ‘The Free Software Definition”

“Free software” does not mean “noncommercial.” A free program must be available for commercial use, commercial development, and commercial distribution.

If you really advocate open source software, the positive participation of the large software companies is a good thing. Let them participate, and only criticize them when they do wrong. Preemptively forking projects to prevent them participating is counter to the OSI license and the philosophies of open source and free software. Using the banner of Free Software to incite exclusion discredits the movement.

As requested by a few fans out there, here are the slides of my presentation:

Pentaho Data Integration 4.0 and MySQL.pdf

I had a great time at the conference, met a lot of nice folks, friends, customers, partners and colleagues. After the conference I was unable to get back home like so many of you because of the Paul Simon singing Eyjafjallajökul volcano in Iceland.

So I ended up flying over to Orlando for a week of brutal PDI 4.0 RC1 hacking with the rest of the l33t super Pentaho development team.  However, after 2+ weeks from home, even a severe storm over Philadelphia couldn’t prevent me from getting home eventually.

Until next time,
Matt

Interesting post by Matt Aslett at the 451 Group. They surveyed over 1000 open source users globally to find out about the drivers behind open source adoption. Interesting stuff.

http://blogs.the451group.com/opensource/2010/04/22/free-451-group-report-on-regional-differences-in-attitudes-to-open-source-adoption/

I wonder how this would manifest itself on the (theoretical) FOSS/GloMM grid?

A good article by Doug Henschen (I got it right this time, Doug) reviewing part of a presentation from Gartner’s BI Summit last week. Here is Doug’s first statement:

The five-year cost of a typical, 500-seat BI deployment ranges from about $150,000 for an open-source system to just over $1 million for the full suite from SAP BusinessObjects, IBM Cognos or Oracle. In between these two extremes are Microsoft, pure-play BI vendors ($522,000 to $674,000) and software-as-a-service (SaaS) BI vendors ($582,000).

The full article is here: http://intelligent-enterprise.informationweek.com/blog/archives/2010/04/gartners_take_o.html

I have stated a few times that the Free Software Foundation (FSF) and its advocates don’t have a vision of the future that I find viable. I have read statements that the best custodians of FOSS are tiny consulting companies, and that Microsoft and Oracle should be barred from participating in FOSS. I don’t see how the software needs of the world can be met by tiny services companies, or how we can magically make the existing market players disappear. But I can’t complain about their vision without providing any vision of my own. So here it is.

The Vision

I subscribe to the theory that a vision is a dream + a plan.

The Dream

Twenty years from now, across the globe, every individual, business, organization, and government entity will have FOSS suitable for all their needs. That is not to say there there is no proprietary software any more – their certainly will be for the next 20 years – just that any and all normal requirements can be met with FOSS.

In this future the notion of intellectual property will still exist, as will software patents (unfortunately). In this future, any software or services company, of any size, whether local or global has the opportunity to participate in the FOSS realm.

We will reach this goal incrementally via an evolution of FOSS software, an evolution of the existing market players, and the creation of new market players.

The Plan 1 – Establish Metrics

Here is my proposal for assessing the state of the dream. By country, we score each software domain in terms of how well FOSS provides suitable solutions that are:

• OSI approved.
• Localized.
• Compliant with all local regulations (accessibility, domain-specific legal requirements etc).
• Stable, usable, and documented.
• Available on multiple platforms (at least 2).
• Available from 3 separate projects (different code-bases), failing that 3 different distros.

Any software under an OSI license is eligible for inclusion – no matter the size of the project, or the business model of provider.

We also score the software with regards to how well it supports all the needs (including support, training, and professional services) of:

• Micro organizations (1-9 people)
• Small organizations (10-99 people)
• Medium organizations (100-250 people)
• Large organizations (> 250 people)

The domains assessed could be (I’m sure there are many more we can add):

Operating System and Middleware
1.    Operating System (OS)
2.    Database (RDBMS)
3.    HTTP and Application Servers
4.    Network Management and Monitoring
5.    Enterprise service bus (ESB), message queue (MQ)
6.    Email
7.    Instant messaging
8.    Calendaring
Horizontal Applications
9.    Customer Relationship Management (CRM)
10.    Locally-compliant Enterprise Resource Planning (ERP)
11.    Content Management Systems (CMS), knowledge base
12.    Call center, case tracking
13.    Ecommerce
14.    Online meeting and conferencing
15.    Voice over IP (VoiP)
16.    Collaboration – forums, wiki etc
17.    Reporting, analysis and Business Intelligence (BI)
18.    Online training
19.    Financial, Budgeting and Planning, including public sector
20.    Distribution
Desktop Applications
21.    Word processing
22.    Spreadsheet
23.    Presentation
24.    Graphics editors
25.    Printing tools
26.    Software and web tools (compilers, editors etc)
Vertical Applications
27.    Healthcare
28.    Education
29.    Government
30.    Agriculture
31.    Insurance
32.    Retail, including Point of Sale
33.    Telecoms
34.    Petrochemical
35.    Pharmaceutical
36.    Travel and hospitality
37.    Engineering, Manufacturing, Construction
38.    Textiles
Obviously, within each of these vertical domains there are multiple applications. Scoring here will be tricky.

System Integrators
39. The local availability of systems integrators that can implement FOSS stacks and solutions.

Scoring is done per country and per domain and is scored from 0 to 9. A score of 0 indicates there is no FOSS option for that domain and geography. A score of 9 indicates the existence of three different FOSS options that meet the needs of large organizations. We can color code by range, red=2 or less, yellow=3 to 6, green=7 or more

2 – Census

We need to find out how close we are to achieving the dream. Volunteer organizations and sponsoring organizations score each domain for a single country, providing notes about the FOSS packages assessed and any services options assessed. The results of the census are publicly available at all times. Academia and analysts could provide much of this data.

3 – Close Gaps

Based on the results of the census, sponsoring organizations provide resources and guidance to help close the gaps. Sponsoring organizations will have many different motivations:

• Wanting a larger local and global market for their services and support offerings.
• Wanting software that is more accessible.
• Wanting more FOSS options in their country or domain.
• Wanting to sell add-ons and extensions.

Or we just allow the natural progress of FOSS to gradually populate the GloMM in a natural  – ‘Game of Life’ / Brownian motion – kind of a way.

4 – Repeat Steps 2 and 3

As time goes by and we repeat steps 2 and 3. The Matrix flushes out gradually, and becomes greener and greener.

Until…

5 – Declare Victory

In my opinion FOSS has won when, and only when, the entire sheet (7000-8000 cells) is lit up in green. At this point the value of FOSS will be clear to everyone. Maybe attitudes towards intellectual property will change then. But we can’t expect them to change before we get close to this point.

Summary

As this gradual global evolution occurs, the existing market players will have to adapt to new market conditions. What they do, and how well they do it, is up to them – but they are welcome to participate. Just because Oracle is now the ultimate custodian of MySQL, does not mean that MySQL should not be listed as one of the FOSS databases. Microsoft, IBM, SAP, Oracle should be accepted into this evolution – whether they survive it is up to them and the global and local markets, not up to anything else.

Any organization that produces FOSS, or localizations, or documentation, or provides services or support for FOSS is deemed to be a friend of GloMM – no matter what their size, history, or business model.

So that’s my vision. I have a defined goal, a way of measuring progress, mechanisms for getting there, and ways for existing market players to participate. I claim it to be reasonable, rational, and viable.

I have no resources at my disposal to execute on any of this. Its just a vision. If only I had a dream + a plan + resources

Steven Levy, who wrote ‘Hackers: Heroes of the Computer Revolution’ twenty five years ago, has re-interviewed some of the original subjects from the book in a Wired article titled ‘Geek Power: Steven Levy Revisits Tech Titans, Hackers, Idealists‘

Its a long read, but worth it. Very interesting.

There are some illuminating quotes from Richard Stallman – twenty fives years ago he said:

I’m the last survivor of a dead culture. And I don’t really belong in the world anymore. And in some ways I feel I ought to be dead.

And today:

In terms of effect on the world, it’s very good that I’ve lived. And so I guess, if I could go back in time and prevent my birth, I wouldn’t do it. But I sure wish I hadn’t had so much pain.

To me, these statements are telling. Why does Stallman feel such pain? I can only think of two possible reasons. Either his ideas and ideals are so far ahead of their time that the world is not ready for them, or they don’t fit well with the world and never will.

His idea that software ‘cannot be owned’ makes no sense to me. The idea that, by sitting at home and writing code on my computer for my own purpose, I am somehow violating the rights of other people is absurd to me. He is denying me the right to my own ideas, and the ability to provide for my family.

Stallman is entitled to his beliefs, to communicate them, and to try to get them adopted. But I’m not surprised he feels pain trying to do this, personally I reject his beliefs. In my mind, I have fewer rights under his world view – I have no right to my own invention, I do not own any software I write, it must be released to the world. He is at least consistent in his beliefs, he feels the same way about music and movies and the companies that try to claim or exert ownership of what they create.

I do think the GPL license, conceptually, it a thing of elegance and beauty. Simple, concise, and powerful. I only have one problem with it – it is self-limiting. It limits usage to those who feel and think exactly one way. It also limits the ability of the established, mainstream market to participate.

It is the same with the Free Software Foundation. They don’t believe in intellectual property. I agree that the software patent system is badly flawed, but I still believe in ownership. But if you don’t comply 100% with their ideals, you are a bad guy. 90% is not good enough. There are no alliances based on common enemies. The FSF has no compelling, or viable, vision of the future that I can see. There is no plan. Just a skirmish war against a unbelieving world. Stallman even objects to the views and actions of Linux’s Linus Torvalds. Meanwhile the markets move on, alliances form, and invention continues. Everything evolves. The world evolves gradually. Unfortunately, Stallman’s views are more creationist – you cannot evolve to them, you will be shut out, berated, and branded as evil until you are 100% there.

Maybe at some time in the future Stallman’s views will prevail. Maybe the world will be a better place when that happens. I just don’t see it happening any time soon.

As can be seen from Levy’s article there is a divide between the hackers: those who view commerce as bad, and those who try to get their ideas adopted widely. Without commerce the world would be one big feudal system – the dark ages again. To reject commerce is to reject the world as it really is. That’s fine for an individual to do, it’s their choice, but to expect the rest of us to do us so in order to adopt or use their ideas is unrealistic.

When we look back in another 25 years at the effect the hackers have had on the world, who’s efforts will have had the biggest impact? The anti-commerce hackers, or the others?

The messages I get from the Windows 7 TV ads are

People who know nothing about computers had to tell Microsoft why Vista sucked so much.

Trust us at Microsoft, we don’t know what we’re doing

We’ll say how bad we are, so you don’t have to

Inspiring.

Mat Aslett at the 451 Group has weighed in on the recent postings and comments between myself and the folks at Nuxeo, in a post – Let he who is without proprietary features cast the first stone

Matt raises some good points in a well balanced approach. Matt comes down on the side of Nuxeo not being open core, but not pure play either:

My own feeling is that Nuxeo’s approach is not open core, since the original definition of open core concerned proprietary products. However, the existence of Nuxeo Studio means that Nuxeo is clearly not 100% open source.

But he doesn’t have a term for what their model is, and is proposing adding a new one to their list:

For that reason, I have come to believe that we need to add a new revenue trigger category to our open source business strategy model, that makes a clear distinction between support subscriptions for 100% open source code, and value-add subscriptions that offer additional hosted services.

As yet, no-one is able to answer my question – if Pentaho stopped offering on-premise deployments and only provided a SaaS offering, would we no longer be open core? Maybe the answer is this new category that Matt mentions.

It looks to me like this model is close to an open core model – just hosted, not on-premise. But the term ‘open core’ does not specify how the non-open parts are deployed, just that some code is open source and some is not. So should we just create sub-categories of open core?

• Open core (on-premise), and open core (hosted)
• Local open core, and remote open core?