𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 කියන්නේ මොනවද??

csharp

Well-known member
  • Nov 19, 2016
    1,263
    781
    113
    Melbourne
    Have you heard the term "𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠" and "𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐬" but have no idea what it means or what they do?? I got you covered!

    👉 What is Data Engineering?


    Data engineering is the primary domain to derive value from collected data. Data engineering imparts practical skills to move data from one location to the other without any loss of information.

    👉 What skills you should have?

    The key skills of a data engineer are to access the data, extract the data to optimally transfer it, and then eventually to be able to store the data. Along with these skills, data engineers should be able to handle very large volumes of data and not lose any value in the process.

    👉 What's so difficult about it?

    A key challenge that comes with handling large data is the loss of context. It's the job of data engineers to interpret the data, which is gathered from various sources, and present a unified view that can be understood by people utilizing that data. Basically, data engineers have to make sense of a never-ending incoming surge of raw data to be able to quickly interpret it, clean it, and then store it in tidy CSV files that are used by data scientists for further analytics.



    large.png


    #dataengineering #data #dataanalytics #bigdata #dataanalysis #dataanalyst #datacleaning #dataengineers #dataengineer
     

    Heshan Daminda

    Well-known member
  • Mar 13, 2009
    46,172
    1
    100,968
    113
    34
    Kalutara
    ඉස්සර ඒ ලෙවල් කරපු ගමන් දැක්ක ඇඩ් එකක් දැක්ක රුක්ෂාන් මාක් සෙට් එකේ රුක්ෂාන් කියන ඩයල් එක ඕඩියෝ ඉංජිනියරින් වලට කෝස් එකක් කරනවා කියල දාල තියෙනවා. දැන් හිතෙනවා අපරාදේ නොගියේ කියල. 😣
     

    csharp

    Well-known member
  • Nov 19, 2016
    1,263
    781
    113
    Melbourne
    Maybe you guys will like this Podcast episode I did with Zack! And if you're into DevOps, You should definitely follow this Podcast!

    17163031-1628432654048-25b1bb542a917.jpg






    ඉස්සර ඒ ලෙවල් කරපු ගමන් දැක්ක ඇඩ් එකක් දැක්ක රුක්ෂාන් මාක් සෙට් එකේ රුක්ෂාන් කියන ඩයල් එක ඕඩියෝ ඉංජිනියරින් වලට කෝස් එකක් කරනවා කියල දාල තියෙනවා. දැන් හිතෙනවා අපරාදේ නොගියේ කියල. 😣
    Yeah I can relate to this as well bng
    ------ Post added on Aug 19, 2021 at 1:36 PM
     

    RandomGuy

    Well-known member
  • Oct 15, 2014
    17,379
    16,222
    113
    ඉස්සර ඒ ලෙවල් කරපු ගමන් දැක්ක ඇඩ් එකක් දැක්ක රුක්ෂාන් මාක් සෙට් එකේ රුක්ෂාන් කියන ඩයල් එක ඕඩියෝ ඉංජිනියරින් වලට කෝස් එකක් කරනවා කියල දාල තියෙනවා. දැන් හිතෙනවා අපරාදේ නොගියේ කියල. 😣

    looking at the person teaching it, giye nethi eka hodai kiyala hithapan.
     

    csharp

    Well-known member
  • Nov 19, 2016
    1,263
    781
    113
    Melbourne
    Bumpss

    wena hoda ewun innawa bn.
    Music Oven palayan :D
    ------ Post added on Aug 20, 2021 at 7:08 PM

    Okay I have the best and simple definition for Data Engineering!

    The simple way of describing a data engineering role is the following workflow -

    Capturing data, storing data, processing data and serving data. Conceptually this is what we're dealing with in data engineering. So don't get too caught up in the tools like Hadoop or Spark, or Scala, or Python, etc. Focus on the concepts before searching for the tools that work for you as a data engineer.

    .
    .
    .

    Original Post
    ------ Post added on Aug 22, 2021 at 12:22 PM