====== Big Data Analytics ======



===== Fundamentals =====


  * [[https://en.wikipedia.org/wiki/Character_encoding|Character encodings]]
    * [[https://en.wikipedia.org/wiki/US-ASCII|ASCII]]
    * [[https://en.wikipedia.org/wiki/ISO/IEC_8859-1|ISO 8859-1]]
    * [[https://en.wikipedia.org/wiki/Unicode|Unicode]]
    * [[https://en.wikipedia.org/wiki/UTF-8|UTF-8]]
    * [[https://en.wikipedia.org/wiki/UTF-16|UTF-16]]
    * [[https://en.wikipedia.org/wiki/UTF-32|UTF-32]]
    * [[https://en.wikipedia.org/wiki/EBCDIC|EBCDIC]]
  * [[https://en.wikipedia.org/wiki/File_format|File formats]]
    * [[https://en.wikipedia.org/wiki/Comma-separated_values|CSV]]
    * [[https://en.wikipedia.org/wiki/XML|XML]] and [[https://en.wikipedia.org/wiki/XML_Schema_(W3C)|XSD]]
    * [[https://en.wikipedia.org/wiki/JSON|JSON]]
  * [[https://en.wikipedia.org/wiki/Database|Databases]]
    * [[https://en.wikipedia.org/wiki/SQL|SQL]]
    * [[https://en.wikipedia.org/wiki/Java_Database_Connectivity|JDBC]]
  * [[https://en.wikipedia.org/wiki/Computer_network|Network]]
    * [[https://en.wikipedia.org/wiki/IPv4|IPv4]] and [[https://en.wikipedia.org/wiki/Subnetwork|network mask]]
    * [[https://en.wikipedia.org/wiki/IPv6|IPv6]]
    * [[https://en.wikipedia.org/wiki/IP_routing|Routing]] and [[https://en.wikipedia.org/wiki/Router_(computing)|Router]]
    * [[https://en.wikipedia.org/wiki/Gateway_(telecommunications)|Gateway]]
    * [[https://en.wikipedia.org/wiki/Firewall_(computing)|Firewall]]
    * [[https://en.wikipedia.org/wiki/Network_address_translation|NAT]]
  * Unix: users, groups, [[https://en.wikipedia.org/wiki/Access_control_list|ACL]]
  * SSH: key authentication, tunneling, SCP




===== Concepts =====


  * [[https://martinfowler.com/bliki/DataLake.html|Martin Fowler on Data Lake]]
  * [[https://knowledgent.com/whitepaper/design-successful-data-lake/|How to Design a Successful Data Lake]]
  * [[https://tidyfirst.substack.com/p/eventual-business-consistency|Eventual Business Consistency (Kent Beck)]]



===== Eco system =====


  * [[http://mattturck.com/bigdata2017/|Big Data Landscape]], [[http://mattturck.com/wp-content/uploads/2017/05/Matt-Turck-FirstMark-2017-Big-Data-Landscape.png|Image]]
  * [[https://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support|Distributions]]
  * [[https://hadoopecosystemtable.github.io/|Eco system]]
  * [[https://www.cncf.io/|Cloud Native Computing Foundation]], Cloud Native Landscape
    * [[https://landscape.cncf.io/|Interactive]]
    * [[https://github.com/cncf/landscape|Static]]
  * [[https://landscape.cd.foundation/|Continuous Delivery Landscape]]
  * [[https://www.thoughtworks.com/radar|Thoughtworks Technology Radar]]



===== Technologies =====


  * Apache Hadoop
    * Apache HDFS
    * Apache MapReduce
    * Apache Parquet
    * Apache Sentry
  * [[https://oozie.apache.org/|Apache Oozie]]: [[https://en.wikipedia.org/wiki/Apache_Oozie|Worklow scheduling system]]
  * Apache Sqoop
  * Apache Hive
  * Cloudera Impala
  * Cloudera Hue
  * Apache Spark with Python, Scala, R
  * Apache Kafka
  * Prometheus
  * Grafana
  * ELK stack
    * Elasticsearch
    * Logstash
    * Kibana
  * Apache Flink
  * Apache Storm
  * Apache Beam
  * [[https://superset.apache.org/|Apache Superset]]
  * [[https://trino.io/|Trino]]



===== Development =====


  * [[https://12factor.net/|The Twelve-Factor App]]
  * Hadoop: [[https://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment|How to set up your development environment]]
  * Spark
    * [[http://spark.apache.org/docs/latest/spark-standalone.html#installing-spark-standalone-to-a-cluster|Installing Spark standalone to a cluster]]
    * [[https://spark.apache.org/developer-tools.html|Spark developer tools]]


===== Open Data =====


  * [[https://atlas.ripe.net/docs/result-streaming/|RIPE Atlas Result Streams]]


===== Miscellaneous =====

  * [[https://seattledataguy.substack.com/p/normalization-vs-denormalization|Normalization vs. Denormalization – Seattle Data Guy]]
  * [[https://www.ibm.com/think/topics/semantic-layer|Semantic Layer]]