DagangNetPOST sat down with Ms Jasbendarjit Kaur, Chief Technology Officer of DNeX recently, and over a cup of teh tarik, her favourite drink, she explained to us what Big Data is and how this technology can affect businesses and organisations.

What is Big Data?
Big data is basically a large, complex data sets coming from various sources and formats. It is also mainly voluminous with high velocity - either real time, near real time or batch - and comes in various formats - either structured or unstructured - which makes traditional data processing even more difficult.

This data if analysed with the right discovery process can bring out value or insights that when tied to business requirements and business intelligence can help businesses and organisations make informed and improved decisions.

When did it first gain prominence in businesses and organisations?
Data has really been in existence for decades, and its collection and storage especially increased with the advancement and capabilities of technologies, which made storing of data becomes easier and cheaper.

Over time, the volume of data built up along with the need to analyse data. The trend has been as such with no particular name given to describe this until possibly in the 2000s when the term Big Data came about.

Who are potential users of Big Data?
There is no limit to potential users of Big Data - it all depends on what data is used and for what purpose. All verticals, anybody who requires insights to make decisions either for diagnostics, proactive or predictive measure can leverage on this technology.

Adoption in Malaysia has largely been focused on sales & marketing, and getting business insights as well as customer relationship management.

Where do we find the right and relevant data to analyse?
It's not about finding the right data to analyse; the use case that you develop is the most important factor.

You need to first define insights or value required based on business objectives and goals before you can develop the relevant use case.

How can you ensure data quality and determine which statistical analysis techniques to apply?
Data quality is all about accuracy in assessing objectives and goals of the business or organisation and what it needs to accomplish.

Once data is assessed, there needs to be data rationalisation and cleansing, and application of customised methodology to provide the right analysis.

The analysis can be descriptive (what is happening), predictive (what will happen next), diagnostic (why did it happen), or prescriptive (what then should I do).

Can you share some tips to implement Big Data? Will this be (gasp) expensive?
You need to first clearly define the business or organisational objectives. Start small for example by implementation on selected priority use case, and grow from there. Continue to evaluate the implementation, and of course, re-align the implementation to the objectives.

The investment will depend on what you want to achieve but by clearly defining the objectives, the investment will be better managed.