Kategoria: Blog posts

Generatiivinen tekoäly on ollut viime vuosina vahvasti esillä julkisessa keskustelussa. Vaikka perinteistä koneoppimista on hyödynnetty yrityksissä jo pitkään, esimerkiksi ChatGPT:n kaltaisten työkalujen ilmestyminen on tuonut tekoälyn laajemman yleisön saataville. Siinä missä aihe aiemmin puhutti lähinnä siihen vihkiytyneitä, on siitä tullut nyt osa myös laajempaa keskustelua. 

Myös teknologiatoimittajien suuret panostukset generatiiviseen tekoälyyn ovat helpottaneet aiheen lähestymistä yhä laajemmalle yleisölle. Samaan aikaan kehitys on ollut huimaa – lähes päivittäin julkaistaan uutisia uusista ominaisuuksista ja työkaluista. Tämä vauhti voi kuitenkin tehdä kokonaisuuden hahmottamisesta haastavaa niille, jotka vasta tutustuvat aiheeseen tai suunnittelevat tekoälyn hyödyntämistä. On vaikea tietää, mistä aloittaa ja mitä mahdollisuuksia ylipäätään olisi käytettävissä. 

Tässä blogissa käydään läpi Snowflaken Cortex AI -ratkaisua, sen tarjoamia mahdollisuuksia sekä joitakin keskeisiä käyttötapauksia. Tarkoituksena ei ole antaa kattavaa kuvausta kaikista ominaisuuksista, vaan ennemminkin nostaa esiin muutamia mielenkiintoisia toiminnallisuuksia ja tarjota hyvä lähtöpiste tarkempaan syventymiseen. 

Snowflake Cortex AI

Snowflake kutsuu tekoäly- ja koneoppimisisratkaisuaan Cortex AI:ksi. Cortex kuuluu kiinteänä osana Snowflakeen, jolloin sen ominaisuudet ovat suoraan käytettävissä ilman erillisiä asennuksia ja konfigurointeja. Tämä helpottaa käyttöönottoa ja itse käyttöä tarjoamalla: 

• Saman skaalautuvuuden kuin Snowflake muutenkin, eli sopeutuu niin pienille kuin suurillekin yrityksille.
• Data säilyy koko ajan Snowflaken sisällä, mikä yksinkertaistaa compliancen ja governancen hallintaa.
• Käyttöoikeuksien määrittely tapahtuu samalla rooleihin perustuvilla ominaisuuksilla, kuin muutenkin Snowflakessa.
• Monia Cortexin toiminallisuuksia voidaan käyttää suoraan Snowsight-konsolista ilman erillistä koodausta, mikä helpottaa varsinkin alkuvaiheessa. 
• Cortex mahdollistaa monia eri tapoja hyödyntää malleja. Cortexissa on useita valmiita malleja eri toimittajilta, joita voi tarvittaessa itse jatkokehittää tai mukaan voi tuoda myös omia malleja.

Valmiit funktiot 

Koneoppisen funktioilla saadaan automatisoitua ennusteiden ja lisätietojen tuottamista. Funktiot on suunniteltu kutsuttavaksi SQLla ja Pythonilla ja niitä voidaan käyttää kiinteänä osana muuta workflowta. Snowflaken AI&ML Studio mahdollistaa funktioiden määrittelyn suoraan myös käyttöliittymästä. Funktiota voidaan käyttää mm. aikasarjoissa, hyödyntäen niitä tietyn arvon ennustamiseen, poikkeamien tunnistamiseen tai luokitteluun. Tällaisia käyttötapauksia voivat olla esimerkiksi: 

• Kysynnän tai myynnin ennustaminen historiallisen datan perusteella 
• Poikkeavuuksien tunnistaminen IT-lokeissa tai taloustapahtumissa 
• Asiakaspalveluun tulevien viestien luokittelu 

LLM funktioita käytetään kirjoitettuun kieleen liittyvissä toimenpiteissä. Funktiot ovat valmiiksi käytettävissä Snowflakessa olevilla malleilla, jolloin käyttöönotto on helppoa ja niitä voidaan kutsua suoraan tai liittämällä jo olemassa oleviin toteutuksiin. Myös nämä funktiot ovat suunniteltu käytettäväksi hyödyntäen SQLlaa tai Pythonia. Funktioita voidaan hyödyntää moninaisissa yhteyksissä, kuten esimerkiksi kielenkäännöksissä, tiivistelmien laatimisessa tai tunnetilan analysoinnissa. Käyttötapauksia LLM funktioiden hyödyntämisessä ovat: 

• Tarkkojen tietojen haku suurista tekstiaineistoista 
• Asiakirjojen, raporttien tai artikkeleiden tiivistäminen 
• Palautteen analysointi positiiviseksi tai negatiiviseksi jatkotoimenpiteitä varten 
• Usealla eri kielellä olevien tekstien kääntäminen samalle kielelle 

Cortex Analyst

Cortex Analyst puolestaan mahdollistaa ei-teknisille käyttäjille tiedon hakemisen luonnollisella kielellä – ilman SQL-taitoja. Se toimii siltana liiketoiminnan ja datan välillä. Cortex Analystia käytettäessä LLM muuntaa käyttäjän kysymyksen SQL kielelle, jolla haetaan vastaus lähteenä käytettävästä strukturoidusta datasta, kuten esimerkiksi tietovarastosta. Vastauksen yhteydessä saadaan myös selite, kuinka kysymys on ymmärretty sekä SQL-lause, josta selviää mistä tieto haettu. Tämä lisää käyttäjäystävällisyyttä vähentämällä väärinymmärrysten määrää. Vinkit kysymyksen muotoiluun sekä ilmoitukset tilanteissa, joissa vastausta ei voida antaa – vaikkapa puutteellisen datan vuoksi – auttavat käyttäjää toimimaan järjestelmän kanssa tehokkaammin ja luotettavammin. 

Kuten muissakin Cortex AI -ominaisuuksissa, myös Cortex Analyst hyödyntää samoja käyttöoikeusrakenteita kuin muualla Snowflakessa. Tämä mahdollistaa helpon ja yhtenäisen oikeuksien hallinnan. Ominaisuus on käytettävissä ulkopuolisista sovelluksista API-rajapinnan kautta, ja se toimii myös yhdessä Snowflaken sisäisen Streamlitin kanssa, jonka avulla voidaan rakentaa nopeasti interaktiivisia sovelluksia. Cortex Analystin käyttötapauksia ovat esimerkiksi: 

• Tiedon kysely tarkemmalla tasolla kuin mitä yrityksen perusraportointi tarjoaa 
• Yksittäisen tiedon hakeminen tietovarastosta tiettyä käyttötapausta varten 
• Erilaiset ad hoc -datatarpeet, joita ilmenee tietovaraston käytön yhteydessä 

Cortex Search 

Cortex Search -hakupalvelua käytetään tietojen hakemiseen laajoista tietomassoista kirjoitetun tekstin avulla. Se on suunniteltu erityisesti strukturoimattoman datan, kuten dokumenttien, käsittelyyn. Hakutoiminnassa hyödynnetään suuria kielimalleja (LLM) ja RAG-tekniikkaa (Retrieve, Augment, Generate), jolloin haun tuloksena syntyy tarkempia ja kontekstuaalisempia vastauksia. Vastaukset perustuvat aina ennalta määriteltyihin lähteisiin, mikä auttaa minimoimaan hallusinaatioiden riskiä. 

Hakupalvelun käyttöönotto onnistuu joko manuaalisesti tai käyttöliittymän kautta hyödyntäen Snowflake AI/ML Studiota. Yksinkertaisimmillaan hakupalvelun pystytys etenee seuraavasti: 

• Valitaan käytettävä kielimalli 
• Määritetään palvelun tallennuspaikka 
• Osoitetaan, mistä datasta vastaukset haetaan 
• Määritetään indeksoinnin aikataulu, jotta vastaukset pysyvät ajan tasalla 

Hakupalvelua voidaan käyttää Snowflaken sisällä käyttöliittymäpohjaisella Cortex Playgroundilla tai – kuten Cortex Analystin kohdalla – myös API-rajapinnan kautta tai Streamlitin avulla. Käyttötapauksia ovat esimerkiksi: 

• Tiedon tarjoaminen kirjoitetun kielen kautta yleisissä käyttötapauksissa 
• Hakumoottorina toimivat chatbotit, joita voidaan hyödyntää esimerkiksi asiakaspalvelussa 
• Yrityksen sisäinen hakupalvelu, jonka avulla työntekijät löytävät nopeasti tietoa esimerkiksi matkustus- tai IT-hankintaohjeista 

Document AI

Document AI mahdollistaa tietojen poimimisen yrityksen strukturoimattomasta datasta (esimerkiksi sopimukset, laskut, kuittien kuvat, lomakkeet tai käsin kirjoitetut asiakirjat) ja tallentaa tiedot strukturoituun muotoon tietokantaan jatkokäsittelyä varten. Tällä ominaisuudella saadaan vähennettyä manuaalista työtä, käsittelyaikaa ja inhimillisiä virheitä. 

Tietojen poimimisen prosessi saadaan myös automatisoitua, jolloin poiminta tapahtuu automaattisesti esimerkiksi uuden dokumentin tultua saataville. Poimittuja tietoja voidaan hyödyntää osana jo olemassa olevaa tietovirtaa. 

Palvelu voidaan määritellä suoraan Snowflaken käyttöliittymästä. Määrittelyvaiheessa käyttäjä esittää kysymyksiä halutuista tiedoista, jolloin malli oppii poimimaan olennaisen sisällön dokumenteista. Mallin opettaminen tapahtuu graafisen käyttöliittymän kautta, mikä poistaa teknisen erikoisosaamisen vaatimuksen ja mahdollistaa liiketoimintaosaajien osallistumisen suoraan prosessiin. 

Poimitut tiedot voidaan tallentaa esimerkiksi tietovaraston tauluihin jatkokäyttöä varten tai hyödyntää osana olemassa olevaa workflow’ta. Document AI -ratkaisua voidaan käyttää esimerkiksi: 

• Käyttötapauksissa, joissa halutaan poimia tietoja dokumenteista strukturoituun muotoon 
• Dokumenttien tarkistuksen automatisoinnissa, kuten allekirjoitusten olemassaolon tunnistamisessa 
• Käsin täytettyjen lomakkeiden tietojen poimimisessa helposti hyödynnettävään muotoon 

Huomioita ennen käyttöönottoa 

Snowflake Cortex AI tarjoaa laajan ja kattavan valikoiman valmiita AI/ML-ominaisuuksia liiketoiminnan tueksi. Käyttöönotto on tehty helpoksi monien sisäänrakennettujen työkalujen ja käyttöliittymien ansiosta. On kuitenkin tärkeää huomioida myös muita näkökulmia ennen ratkaisujen viemistä tuotantoon – erityisesti dataan liittyviä. Alla muutamia keskeisiä huomioita: 

• Käyttötapaukset: Käyttötapausten huolellisella suunnittelulla on keskeinen rooli. Mitä ongelmaa ollaan ratkaisemassa? Mikä on tavoiteltu lopputulos? Mitkä ovat sen vaikutukset, hyödyt ja liiketoiminnalliset tavoitteet? 
• Data-arkkitehtuuri: Tukeeko nykyinen data-arkkitehtuuri AI/ML-ratkaisujen käyttöönottoa? Kuinka pitkälle voidaan hyödyntää olemassa olevia rakenteita? Tarvitaanko muutoksia, jotta arkkitehtuuri tukee tekoälyn ja koneoppimisen tarpeita? Ideaalisti AI/ML on yksi datan hyödyntäjistä ja tuottajista muiden joukossa. 
• Datan ja sen laatu: Onko tarvittava data jo olemassa tai hankittavissa käyttötapaukseen? Mikä on datan nykyinen laatu? Tarvitaanko toimenpiteitä, jotta data täyttää laatuvaatimukset? Laadukas data on onnistuneen tekoälyratkaisun perusta. 
• AI on yhtä hyvä kuin datasi! 

-Asko Ovaska 

Etlia on toteuttamassa useita Snowflake-ratkaisuja eri asiakkaille – mikäli kiinnostuit tai haluat kuulla lisää, ota rohkeasti yhteyttä!

The Importance of Reliable Sustainability Information

Don’t be mistaken, whether corporate sustainability reporting is mandatory now or later (some of the CSRD requirements may be postponed by the recent European Commission’s Omnibus package proposal), strategic sustainability areas constitute priorities.

For the ones of you fluent in CSRDish, the Esperanto of the sustainability professionals community, we are talking about the “metrics related to material sustainability matters”.

There is an indisputable need for reliable information on the sustainability performance, regardless of the level of integration of sustainability in companies and the reporting requirements in force. Responsible data-driven decision-makers demand information they can trust.

Challenges in Sustainability Reporting

As a sustainability dinosaur and an ex-PwC Sustainability Reporting Assurance manager I happen to have a few hints on what it takes to build trust in sustainability information, here are some!

Let’s play a little game together, shall we? Go through the few situations below where people are using information on a company’s sustainability performance and ask yourself whether it matters that the information is accurate. Keep count.

• You are looking at the energy intensity performance of the past year on your company’s intranet’s report to determine whether all employees will receive a bonus as planned by the incentive programme of your company
• A potential client visits your factory and asks you about the number of days with zero work accidents presented on the shop floor’s dashboard
• You were asked by the top management to propose ambitious but realistic short-term GHG emissions scope 3 reduction targets, you look at the past 5 years performance published in the company’s voluntary sustainability report
• A retailer, who is a strategic client to your company has set new procurement requirements and you have just a few weeks to provide evidence that the materials used in the packaging of your products are sustainably sourced.

How many did you get? And most important, did you know whom to turn to find out? Did you have any doubts about the calculation methods, the data quality or the results altogether? How would you make sure the data is up to date?

Behind all the situations above, there is a reporting process be it explicit or not. Therefore, solutions look pretty much the same for sustainability reporting than for others and assurance procedures follow the same standards too. But there is just this little twist more, that makes it so much more fun to play around with: a multitude of calculation methods, sources of raw data, the use of estimates and the fact that there is a relatively short history of mandatory assurance.

Ensuring Data Quality and Streamlining the Reporting Process

Here are some tips to get your pulse down and a confident smile back on your face:

• Data quality: establish procedures to ensure robust data is used.
  • Remember the S*-in-S*-out principle? Find out what your KPIs are built upon, where the raw data are originating from and whether you can tell for any given KPI, what set of data was used.
    • Draw the flow of information, this will probably look like a very large family-tree if you are dealing with GHG emissions scope 3 data!
  • Manual manipulation is sadly still common practice (someone looks up the value from a screen, writes it on a piece of paper and types the figure into a worksheet’s cell or a second person types values into the body of an e-mail that is sent to a person who also uses manual input methods), things can go wrong at each and every turn and if you repeat this over a few thousands of figures…
    • Seriously consider automating your reporting process. To find out more, reach out to professionals with proven-track records of ESG automation such as Etlia
  • Find out what assumptions are made, are the figures based on estimates, are they based on measured or calculated information, what calculation methods are used. Was it hard to check this bit?
    • Implement a well-documented, well-maintained and user-friendly reporting process
• Shake your reporting process’s tree (I know I keep talking about trees, bear with me…) and find out how robust it is:
  • double-check, re-calculate
  • walk-through the process, try and follow the trail all the way up to the raw data
  • use sensitive analysis tools,
  • meet the people involved in reporting, are they aware of the role they play? do they know what the information they process is used for and by whom?
• Motivate your reporting team:
  • engage people affecting the quality of your information, explain how valuable their contribution is and listen to what they can teach you on reporting, they know their stuff!
  • clean it up: make sure sources of errors are addressed and no one is blamed for them, it is a collaborative effort
  • celebrate, there is no such thing as a small victory! Make improvements every time they count. Don’t wait for the big solution to solve all your problems. Tools do not create a reporting process, they only facilitate it.
  • sometimes it can be hard to give up on old ways of doing things, ask your quality colleagues or your change management gurus for tips
  • lean your reporting process: aim at a smooth, tidy, efficient and quality data producing process!

Etlia and Luotsi Yritysvastuupalvelut

Combining the expertise of the Etlia data engineer expertise and Luotsi’s deep understanding in sustainability reporting requirements and processes these companies provide together a robust framework and solution for organizations to navigate the complexities of sustainability reporting and make informed, data-driven decisions.

If you need more information, please contact adeline@yritysvastuupalvelut.fi or fill the contact form on our website.

– Adeline Maijala, CEO, Luotsi Yritysvastuupalvelut Oy – Etlia’s Co-Champion

With the ongoing wave of SAP S/4HANA implementations, many organizations are rethinking their data and analytics portfolios. At Etlia Data Engineering, we frequently help businesses navigate these decisions. When it comes to analytics with SAP S/4HANA, the choices often resemble a 1X2 football bet. Here’s a short practical breakdown of the choices:

1: All-in on SAP (Pure SAP)

Choosing ”1” means relying entirely on SAP’s built-in tools like Datasphere and SAP Analytics Cloud (SAC).

Pros: 

– Seamless integration across SAP systems with optimized performance 
– Real-time insights and SAP’s own functionalities (e.g. AI applications and planning) tied to business processes 
– Simplified vendor management with a single tech stack 

Cons: 

– Limited flexibility 
– Dependence on SAP’s offering and innovation timeline 
– Scarcity of SAP analytics experts 

This option is ideal for businesses prioritizing simplicity and full integration with SAP ERP.

X: The hybrid play 

The ”X” approach combines SAP tools with external platforms like Azure and Databricks, blending the best of both worlds. 

Pros: 

– Flexibility and scalability 
– Access to advanced AI and machine learning capabilities 
– Retains some SAP-native advantages 

Cons: 

– Risk of data silos and duplication 
– Complex governance and skill requirements 
– Higher operational complexity and TCO 

This hybrid model works best for organizations seeking flexibility while maintaining ties to SAP ERP. This is the most complex scenario with the highest total cost of ownership (TCO), so it’s essential to carefully assess the business case to justify the additional investment. Be sure to identify the specific reasons and value drivers that make this approach the right choice for your organization. 

 2: External Data Tools and Platforms (Non-SAP) 

Selecting ”2” involves moving all analytics to external platforms such as Azure, AWS, Snowflake, or Databricks. 

Pros: 

– Unmatched scalability, flexibility, and customization 
– Wide support for cutting-edge tools 
– Independence from SAP’s constraints 

Cons: 

– Greater difficulty integrating with SAP ERP 
– Higher management overhead for cross-platform data 
– Dependence on non-SAP experts 

This option suits organizations focused on top-tier analytics and innovation, even if it means operating outside the SAP ecosystem.

Key considerations for your analytics strategy on top of S/4 HANA 

1. Align analytics to business needs

– If seamless process integration and simplicity are priorities, SAP-native solutions are a strong starting point. 
– For advanced analytics or scalability, consider hybrid or external approaches. 

2. Evaluate SAP’s analytics offering  

For organizations already committed to SAP S/4HANA, it’s logical to start with SAP’s integrated tools like Datasphere and SAC. SAP is also investing heavily in developing advanced business AI capabilities that integrate seamlessly with SAP’s own tech stack. SAP data solutions are designed to function together with S/4HANA simplifying deployment and accelerating ROI.  

3. Don’t overlook Best-of-Breed solutions 

While SAP’s analytics tools are rapidly maturing, platforms like Microsoft (Azure, Fabric), AWS, Databricks, and Snowflake may provide more advanced AI and ML capabilities. Ensure you have a robust approach for any SAP data extraction e.g. by using SAP Datasphere and be aware of potential challenges and limitations when integrating non-SAP solutions with S/4HANA such as restricted external data extraction (e.g. SAP Note 3255746).  

The winning strategy for SAP S/4HANA analytics 

The choice between SAP-native, hybrid, and external solutions depends on your organization’s infrastructure, data strategy, and goals. Start by evaluating SAP’s analytics tools, as they’re optimized for S/4HANA. For advanced functionality or flexibility, explore hybrid or non-SAP options. 

Stay tuned for upcoming blogs, where we’ll dive deeper into each scenario to help you make informed decisions.

 Interested in learning more or discussing your specific needs? Book a meeting with us today! 

At Etlia Data Engineering, we’ve partnered closely with our clients to develop efficient, automated data pipelines that streamline ESG reporting. As ESG reporting becomes a mandatory part of corporate responsibility, businesses face growing pressure to provide precise and transparent data. By leveraging Databricks for CO2 emissions reporting and Power BI for visualization, we create seamless solutions that offer valuable insights to support decision-making.

The Challenge: Moving away from manual processes

Carbon footprint reporting is becoming an essential part of every corporate ESG disclosure. However, for many organizations, the process is still labor-intensive, involving manual data collection, entry, and calculations. Automating this process significantly reduces errors, improves accuracy, and saves time, but it requires the right strategy and tools. Here’s how we tackled this challenge.

1. Defining your reporting targets:

Before you begin automating, it’s important to have a clear understanding of your reporting goals. At Etlia, we set up our clients’ systems to handle overall and granular-level CO2 calculations. This allows them to drill down into emissions from specific equipment components, logistics emissions, supplier emissions, or even individual processes, identifying the most impactful contributors to their overall carbon footprint.

2. Assessing your data and data sources:

The quality of your carbon footprint reporting is only as good as the data behind it. Therefore, evaluating your data sources is critical. In many cases, organizations need to pull data from multiple systems—ERP, Factory data, common coefficient external data, energy management systems and supplier data sources to get a full picture. To ensure data accuracy and reliability, we conduct a thorough assessment of your existing data sources, identifying potential gaps and inconsistencies. This assessment helps us determine the most appropriate data collection and integration methods to optimize your carbon footprint reporting.

3. Selecting the right technology stack:

Usually, it makes sense to follow your organizations’s architecture and technology guidelines for any new data domains. At Etlia we have experience of building data pipelines with most of the leading technologies.  

In our experience e.g. Databricks is a good choice as the backbone of data processing due to its ability to handle large volumes of structured and unstructured data. Databricks gives the flexibility to model the complex hierarchical data structure using PySpark, helped to speed up the development of the pipeline 

For visualization we usually recommend Power BI as the infrastructure is well fit within Azure framework commonly used by Finnish organizations. Once the data is processed and the carbon footprint contributors identified, Power BI enables clear, interactive dashboards that stakeholders can easily interpret and act upon.

4. Data modelling for CO2 calculation:

At the core of our solution is a hierarchical data model that supports multi-level CO2 emission calculations. This model allows for both high-level overviews and granular insights into specific emission sources. We integrate external datasets for CO2 emissions factors, ensuring that the data model could adjust automatically as new data was ingested. It is very likely that other tools may also be used in parallel, and our solution is designed to seamlessly integrate with these tools, providing a comprehensive and flexible approach to CO2 emission management.

5. Developing the solution: start with an MVP:

One of the key lessons we have learned is the importance of starting small and scaling over time. We usually begin by developing a Minimum Viable Product (MVP), focusing on automating a single reporting process. This helps us to identify the dependencies, missing data sources and required stakeholders to productionize the pipeline. 

The MVP approach allows our clients to see immediate benefits of reduced manual workload and improved data accuracy while keeping the project manageable.

6. Continuous improvement and scaling the system:

Once your MVP is successful, you can work on gradually expanding the system’s capabilities. This includes integrating additional data sources, refining the data model, and enhancing the Power BI dashboards with more sophisticated analysis and forecasting capabilities. As the system scales, so do the benefits, enabling more comprehensive and actionable CO2 reporting. 

Implementing automated carbon footprint reporting provides considerable long-term benefits, enabling organizations to fulfill their ESG commitments more efficiently while also saving time and minimizing errors. From our experience, modern tools like Databricks and Power BI significantly streamline and improve the reporting process. Whether you’re beginning or seeking to enhance your current system, automation is essential for effective and precise CO2 reporting.

–Raaju Srinivasa Raghavan

Discover the benefits of automating your ESG data pipeline in our latest blog.

Interested in taking the next step? Contact us to discuss how we can help automate your ESG reporting processes.

Why automate your ESG data pipeline and how to do it?

While requirements for ESG reporting for businesses are tightening many organizations are still struggling with inefficient manual reporting processes that compromise the quality and assurance-readiness of ESG reporting.

It is not always easy to find actual data for ESG KPIs – hence manual data input and calculation logic based on e.g. emission factors, averages and standard rules will be reality for some parts of ESG reporting also in the near future.  

Based on our experience, organizations can improve their reporting process significantly by gradually automating ESG data pipelines wherever possible – this brings immediate benefits by improving the efficiency of the reporting process as well as allowing better accuracy of your ESG reports and transparency into underlying data. 
 
At Etlia Data Engineering we have successfully implemented automated ESG data pipelines for our clients and in this blog, we dissect our key learning points based on our experiences. 

Why consider automating your ESG data pipeline? 

Main benefits our customers have achieved by automating their ESG data pipeline: 

• Transparency and assurance-readiness: Automating data pipeline from operative systems helps ensure ESG reports comply with regulatory requirements and provide audit trails for accountability and transparency. 

• Cost optimization: Reducing the need for manual entry of ESG data, for example using Excel files lowers labor costs and minimizes the cost impact of errors and delays. 

• More up-to-date ESG reports: Automation significantly reduces the time required to gather, process, and update data, enabling real-time or near-real-time reports allowing management to take action faster than with manual process. 

• Superior data quality: Automated ESG data pipeline is remarkably less error-prone compared to manual processes.  

• Scalability: An automated ESG data pipeline can scale-up and handle increasing volumes of data as the company grows, unlike manual processes that struggle to scale efficiently. 

What are the biggest challenges? 

The most common hurdles our clients are facing when building ESG data solutions: 

How to do it? 

These are our recommended steps to automate your ESG data pipeline 

• Get started: The sooner you start building automated data flow from operative systems the better it will be for managing the overall roadmap, as it will take time and substantial investments. It is best to get started and move away from manual processes gradually. 

• Build your understanding: Understanding of the KPIs and ESG reporting requirements such as EU CSRD is crucial, as they help to define the data needed to build the ESG pipeline.  

• Define targets: Define stakeholders’ targets and roadmap for your ESG reporting development.  

• Assess your data and data sources: First, define the data you can get from internal sources and whether there is a need for external data. A good example in the case of the process industry could be that you need material information from suppliers and external data for the coefficient from other providers. The exercise of understanding source data and systems helps to determine if you could stay with existing data architecture or do you need a new one to support the ESG pipeline. 

• Select technologies: Choosing the right platform for your ESG data is crucial considering the maintainability and complexity of data sources. You may be attracted to use tools that have fancy pre-defined templates but be aware, 1) this does not remove the need for having a proper data platform and 2) these tools might have other limitations such as very specific requirements for overall architecture that could be in conflict with your organization’s guidelines. 

• Data modelling: Start with an analysis identifying how much data is available to build your ESG pipeline. Data modeling for ESG will require combining the data from your systems with reference data (for common data and coefficients) to calculate your emissions and other KPIs. You should expect the model could probably contain hierarchical traversing to calculate the emissions on all granularities to identify which is the major contributor, and this could also be a decider in choosing your architecture. 

• Solution development: Ideally the development process should follow your organization’s common process for building data solutions. At Etlia Data Engineering we always recommend agile development methodologies.  

• Gradual development: Start Small. Due to the complex nature and limited availability of the data it’s a good approach to proceed modularly and build your solution step by step automating one part of the data flow at a time.  

– Raaju Srinivasa Raghavan & Mikko Koljonen 

Are you ready for ESG data automation? If you have any questions or need support in your ESG data process don’t hesitate to reach out to us by booking a short meeting!

Along with the current emergence of AI there is also a lot of excitement about “Business AI” or alternatively “Enterprise AI”. Although there is no single definition of Business AI, it can be seen as business processes and decision making supported by various AI tools often embedded into enterprise software products.

While generative AI solutions like GPT and various “co-pilot”-types of AI assistants are very usable for some use cases we are still some steps away from fact-based AI-supported company or business unit-wide decision making that relies on hard quantitative business data. Currently, the focus of business AI use case development is mainly on creating new types of user interfaces and supporting specific business process workflows where the new generative AI models have a competitive advantage. But when asking your internal AI assistant to provide you with a report on company KPI’s you have a substantial risk of getting wrong results, unless your underlying data is reliable. Quantitative data is still often leveraged by the conventional ML algorithms and some organizations are championing this very well – some have been doing this for a few decades already!

In the current buzz it is easy to forget that one of the biggest challenges is that you cannot fully rely on generic generative AI models to answer factual questions correctly in a business context. Leading software companies, such as Microsoft, Salesforce and SAP, are currently pouring their resources into Business AI solutions designed to take your business to new heights. While AI assistants and automated workflows are useful tools, running a business successfully demands a thorough understanding of business logic and trust in underlying numbers. It is easy to forget that business AI needs data. So how to make your analytics data assets ready for business AI? Let’s find out!

More than ever the key question is the quality of the data. You do not want to have a Business AI solution that uses wrong data as a basis for the desired outcome.

The only way to build working business AI solutions is to enhance your models based on CORRECT business data. How to achieve that? Where to get that correct business data? Answer is simple – you need to start by taking care of the impeccable data flow in your data pipelines. Unless the correct data is available for the AI models you will be in trouble.

High-quality data is a daydream for anyone dealing with massive corporate business data solutions, often struggling with data integrity. An optimist might say that Business AI is pushing us to a new era where we will finally have the single version of the truth.

Here is my take on the top 10 activities that everyone should be doing today to make their data assets and organization ready for business AI:

1. Get started: cultivate an AI mindset and understanding by training people and start to use available AI tools such as AI-assistants
2. Assess and understand your current data and systems
3. Set your ambition level and goals based on business strategy and targets
4. Invest in skills: own and external
5. Plan your roadmap and high-level data architecture based on your ambition level and possible use cases
6. Ensure adequate data governance within your organization
7. Select technologies that suit your overall IT systems landscape
8. Design your detailed data architecture and solutions properly to avoid surprises
9. Build a sustainable and modern data architecture to allow impeccable flow of data from source to your business AI solution
10. Don’t forget: continuous housekeeping and incremental development based on your roadmap

As a business or IT leader you surely want to get started today to stay in the game and ensure your data architecture drives your organization’s future success. Make sure your data assets are ready for business AI solutions, and follow our step-by-step tips!

Etlia is a fast-growing and focused data engineering company specializing in business data. If you are interested in learning how to build your data pipelines business AI ready don’t hesitate to get in touch by booking a meeting with us.

Book a meeting or contact us!

– Mikko Koljonen

In today’s fast-paced work environment, it’s easy to get caught up in deadlines, targets, and the daily grind. But sometimes, amidst the hustle, we forget something crucial: appreciation.

In the end people matter – hence one of our key values at Etlia is “We appreciate people”. Naturally this value encompasses all the essentials such as appreciating people irrespective of race, sex, religion, cultural background and age. But appreciation is much more than that: taking the time to acknowledge and celebrate the contributions of our colleagues is essential for building a positive, thriving workplace.

Why Appreciation Matters?

Appreciation isn’t just a feel-good nicety; it has a tangible impact on our work lives. Studies show that employees who feel valued are:

• More engaged: When we feel our efforts are recognized, we’re more likely to go the extra mile and be invested in our work.  
• More productive: Appreciation fosters a sense of purpose and motivation, leading to increased productivity.  
• More collaborative: When appreciation is expressed, teams feel a sense of unity and are more likely to work together effectively.  
• Less likely to leave: Feeling valued contributes to employee satisfaction and retention, reducing turnover.

Appreciation in Action at Etlia:

• We appreciate people irrespective of race, sex, religion, neurodiversity, cultural background and age.  
• We celebrate people. We celebrate successes and life milestones by rewarding employees with small gifts for their achievements and the joyful news in their lives. 
• We recognize people’s contributions. Etlian’s contributions to Etlia or Customers are recognized on Etlia’s weekly meetings and appreciated in the communication channels. Also, they are rewarded according to the level of achievement.  
• All Etlians helping with recruitment are rewarded. We encourage every employee to actively participate in shaping our team and culture. 
• All Etlians getting certified in relevant technologies are recognized and rewarded in Etlia.

The Bottom Line

Taking the time to appreciate our colleagues isn’t just the right thing to do; it’s a smart business decision. By fostering a culture of appreciation, we create a more positive, productive, and successful workplace for everyone!  

At Etlia we are building the best community and platform for top experts’ professional growth.

– Raaju Srinivasa Raghavan

Interested to join Etlia’s growing team of champions – get in touch and let’s meet for a coffee!

Analytiikan tehokas hyödyntäminen on nykyliiketoiminnassa välttämätöntä, mutta alkuun pääseminen voi tuntua hankalalta. Microsoft Fabric on analytiikka- ja data-alustaratkaisu, joka yhdistää useita Microsoftin data- ja analytiikkatyökaluja yhdeksi kokonaisuudeksi, tehden datan käsittelystä ja hyödyntämisestä tehokkaampaa. Sen Direct Lake -ominaisuus mahdollistaa datan nopean päivityksen ja käsittelyn, mikä auttaa raportoinnin kehittäjiä ja nopeuttaa liiketoiminnan päätöksentekoa. Videolta opit, miten Microsoft Fabric otetaan käyttöön käden käänteessä ja kuinka luoda Direct Lake -raportti vaivattomasti.

Miksi rakentaa Direct Lake?

Delta Laken Parquet-tiedostot mahdollistavat saumattoman integraation Direct Lake tietomallin kanssa, mikä poistaa tarpeen tietojen erilliselle siirrolle Power BI:hin. Lisäksi Direct Lake on optimoitu skaalautuvaksi ja se pystyy käsittelemään tehokkaasti suuria määriä tietoja. Tämä tekee siitä ihanteellisen valinnan suurien raportointikokonaisuuksien hallintaan Lakehouse -ympäristössä.

Opi hyödyntämään Microsoft Fabricin tarjoamia mahdollisuuksia liiketoiminnassasi ja aloita analytiikan tehokas käyttö tänään. Varaa ilmainen puolen tunnin kartoitus asiantuntijoidemme kanssa!