lrnz.name

User Tools

Site Tools

Trace: • dokuwiki • playground • python • development • software • big-data-analytics • ssh
big-data-analytics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
big-data-analytics [2018-01-24 15:40] – mrksbig-data-analytics [2026-04-22 10:54] (current) – mrks
Line 1: Line 1:
 ====== Big Data Analytics ====== ====== Big Data Analytics ======
 +
 +
 +
 +===== Fundamentals =====
 +
 +
 +  * [[https://en.wikipedia.org/wiki/Character_encoding|Character encodings]]
 +    * [[https://en.wikipedia.org/wiki/US-ASCII|ASCII]]
 +    * [[https://en.wikipedia.org/wiki/ISO/IEC_8859-1|ISO 8859-1]]
 +    * [[https://en.wikipedia.org/wiki/Unicode|Unicode]]
 +    * [[https://en.wikipedia.org/wiki/UTF-8|UTF-8]]
 +    * [[https://en.wikipedia.org/wiki/UTF-16|UTF-16]]
 +    * [[https://en.wikipedia.org/wiki/UTF-32|UTF-32]]
 +    * [[https://en.wikipedia.org/wiki/EBCDIC|EBCDIC]]
 +  * [[https://en.wikipedia.org/wiki/File_format|File formats]]
 +    * [[https://en.wikipedia.org/wiki/Comma-separated_values|CSV]]
 +    * [[https://en.wikipedia.org/wiki/XML|XML]] and [[https://en.wikipedia.org/wiki/XML_Schema_(W3C)|XSD]]
 +    * [[https://en.wikipedia.org/wiki/JSON|JSON]]
 +  * [[https://en.wikipedia.org/wiki/Database|Databases]]
 +    * [[https://en.wikipedia.org/wiki/SQL|SQL]]
 +    * [[https://en.wikipedia.org/wiki/Java_Database_Connectivity|JDBC]]
 +  * [[https://en.wikipedia.org/wiki/Computer_network|Network]]
 +    * [[https://en.wikipedia.org/wiki/IPv4|IPv4]] and [[https://en.wikipedia.org/wiki/Subnetwork|network mask]]
 +    * [[https://en.wikipedia.org/wiki/IPv6|IPv6]]
 +    * [[https://en.wikipedia.org/wiki/IP_routing|Routing]] and [[https://en.wikipedia.org/wiki/Router_(computing)|Router]]
 +    * [[https://en.wikipedia.org/wiki/Gateway_(telecommunications)|Gateway]]
 +    * [[https://en.wikipedia.org/wiki/Firewall_(computing)|Firewall]]
 +    * [[https://en.wikipedia.org/wiki/Network_address_translation|NAT]]
 +  * Unix: users, groups, [[https://en.wikipedia.org/wiki/Access_control_list|ACL]]
 +  * SSH: key authentication, tunneling, SCP
 +
  
  
Line 7: Line 38:
  
   * [[https://martinfowler.com/bliki/DataLake.html|Martin Fowler on Data Lake]]   * [[https://martinfowler.com/bliki/DataLake.html|Martin Fowler on Data Lake]]
 +  * [[https://knowledgent.com/whitepaper/design-successful-data-lake/|How to Design a Successful Data Lake]]
 +  * [[https://tidyfirst.substack.com/p/eventual-business-consistency|Eventual Business Consistency (Kent Beck)]]
  
  
Line 16: Line 49:
   * [[https://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support|Distributions]]   * [[https://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support|Distributions]]
   * [[https://hadoopecosystemtable.github.io/|Eco system]]   * [[https://hadoopecosystemtable.github.io/|Eco system]]
 +  * [[https://www.cncf.io/|Cloud Native Computing Foundation]], Cloud Native Landscape
 +    * [[https://landscape.cncf.io/|Interactive]]
 +    * [[https://github.com/cncf/landscape|Static]]
 +  * [[https://landscape.cd.foundation/|Continuous Delivery Landscape]]
 +  * [[https://www.thoughtworks.com/radar|Thoughtworks Technology Radar]]
  
  
Line 27: Line 65:
     * Apache Parquet     * Apache Parquet
     * Apache Sentry     * Apache Sentry
 +  * [[https://oozie.apache.org/|Apache Oozie]]: [[https://en.wikipedia.org/wiki/Apache_Oozie|Worklow scheduling system]]
   * Apache Sqoop   * Apache Sqoop
   * Apache Hive   * Apache Hive
Line 33: Line 72:
   * Apache Spark with Python, Scala, R   * Apache Spark with Python, Scala, R
   * Apache Kafka   * Apache Kafka
 +  * Prometheus
   * Grafana   * Grafana
   * ELK stack   * ELK stack
Line 41: Line 81:
   * Apache Storm   * Apache Storm
   * Apache Beam   * Apache Beam
 +  * [[https://superset.apache.org/|Apache Superset]]
 +  * [[https://trino.io/|Trino]]
  
  
Line 47: Line 89:
  
  
 +  * [[https://12factor.net/|The Twelve-Factor App]]
   * Hadoop: [[https://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment|How to set up your development environment]]   * Hadoop: [[https://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment|How to set up your development environment]]
   * Spark   * Spark
Line 52: Line 95:
     * [[https://spark.apache.org/developer-tools.html|Spark developer tools]]     * [[https://spark.apache.org/developer-tools.html|Spark developer tools]]
  
 +
 +===== Open Data =====
 +
 +
 +  * [[https://atlas.ripe.net/docs/result-streaming/|RIPE Atlas Result Streams]]
 +
 +
 +===== Miscellaneous =====
 +
 +  * [[https://seattledataguy.substack.com/p/normalization-vs-denormalization|Normalization vs. Denormalization – Seattle Data Guy]]
 +  * [[https://www.ibm.com/think/topics/semantic-layer|Semantic Layer]]
  
big-data-analytics.1516804829.txt.gz · Last modified: (external edit)