Data Collection

How data is collected?

1. Collection

• Sources: Big data flows in from diverse origins:
  • Internal: Company databases, transaction records, customer interactions.
  • External: Web scraping, social media feeds, IoT devices, public datasets.
• Methods:
  • Batch Processing: Gathering data in large chunks at scheduled intervals (e.g., daily or weekly).
  • Real-time Streaming: Continuous capture of data as it's generated (e.g., financial market data, sensor readings).

Collect

Quantitative data Qualitative data