Big Data-Future Has Arrived
Due to arrival of new technologies and social networking sites, the amount of data produced is growing rapidly every year. By 2003, the amount of data produced was 5 million gigabytes. The same amount of data was produced every 10 minutes in 2011 and the amount is still increasing on a large scale.
Big Data is a collection of large data sets which can not be processed using a traditional approach. It requires various tools, techniques and frameworks to process the huge amount of data.
Less amount of data produced can be managed using the traditional approach of data handling. Here data will be stored in RDBMS(Relational Database Management System) like Oracle Database, MS SQL Server, MySQL or DB2.
This traditional approach of manipulating data can be useful with a smaller volume of data but dealing with large volume of data is painful. Minimum size of Big Data starts with at least 1 TB.It is about a TB or ZB (Zettabytes) of file.
From 1975-80’s use of RDBMS-Fixed schema,Fixed rows & Tables were there and it was carried out till 2000,then Facebook, Linkedin, Twitter started coming into picture. Earlier the information stored was UNSTRUCTURED.
UNSTRUCTURED data comprises of different format of files such as:
Audio files,video files,posts,images etc).In Year 2004-05,the data started increasing and it is when the amount of unstructured data increased & is still growing.
The 4 V’S of Big Data By IBM
- Volume: Large Volume Of Data
- Variety: Different formats(Audio,Video,Images,Posts,etc)
- Velocity: Speed of Data Processing.
- Varacity: To check that the Data is Genuine.
The traditional approach of data processing works on a portion/sample of data but Big Data works on the whole of any Data(collection of datasets).
Verticals that come under the roof of Big-Data
-
Stock Exchange: Data produced in stock exchange are huge enough to be handled by the traditional approach of Data handling(RDBMS).Here Buy/Sell decisions are made on share of different companies,Thus producing a big stack of data.Here Big-Data Techniques and tools can be used to overcome the disastorous data.
