Everything concerning your business’s past and current status is recorded as bits of data. Marketing numbers, human resources, company budgeting, sales volumes — you name it. The number of business domains the data comes from can be large. But, as a business, you might be interested in extracting value from this information instead of just collecting it. Be thankful that there is software for everything.
Business intelligence (BI) is a set of technologies and practices for transforming business information into actionable reports and visualizations. Established in the early 80s and developed over time as a separate industry, BI gave birth to numerous roles and professions. We have already explained the role of an ETL developer. This material uncovers the specifics of the underlying BI data infrastructure, so we suggest you read it to gain deeper insight on the topic.
Today we’ll discuss the role of a BI developer, their typical responsibilities, background, and required skills. We’ll also define the difference between other standard roles involved in building BI systems and specific cases that require you to hire a BI developer.
What is a business intelligence developer?
A business intelligence developer is an engineer that’s in charge of developing, deploying, and maintaining BI interfaces. Those include query tools, data visualization and interactive dashboards, ad hoc reporting, and data modeling tools. But, as we are speaking about business intelligence, we need to give a stricter explanation of this technological concept.
BI system divided by layers
The whole BI system can be divided into its inner part and the actual user interface to interact with data. Why is this division so important? Independent of the system’s complexity, a typical BI platform has 3 layers: a data source layer, a warehouse layer, and a reporting layer. Let’s break them down:
The data source layer is where the raw data is stored. Those are any of your databases, cloud storages, and separate files filled with unstructured data. Data sources are the starting points of any BI system because they are connected with all the following data-integration tools, storages, and business intelligence UI.
The warehouse layer includes all the technologies that facilitate the warehousing process. These are both unified storage for all the corporate data and tools performing Extraction, Transformation, and Loading (ETL). The data is moved to a single database for a few reasons:
- to create a single version of truth,
- to access all the business data from one place, and
- To standardize data into coherent formats so that it can be queried by users.
So, data source and warehouse layers are considered an underlying infrastructure of the BI system. Building and maintaining it is a field of responsibility for database/ETL developers and data analysts/engineers.
The reporting layer is the final point for data. This layer is the actual BI interface that allows users to access data, dragging it from a warehouse for analysis. Analytical capabilities may vary depending on business requirements for a system. But in the most basic scenario, those are SQL queries and simple data visualization instruments.
An example of a BI dashboard showing graphs and scores as data as data viz
Source: docs.microsoft.com
Every piece of software that makes up the reporting layer is the responsibility of a business intelligence developer. In the scope of a business intelligence project, a BI developer takes engineering, management, and strategic planning responsibilities. So, this person can be both a project lead and/or an executor.
As we have defined the place of a BI developer in the scope of the whole BI platform, let’s take a look at this person’s responsibilities.
What are the responsibilities of a BI developer?
The project scope defines the degree of involvement for a certain role, as engineers with similar technology stacks and domain knowledge can be interchangeable. Developing BI interfaces requires deep experience in software engineering, databases, and data analysis. So, partially, data engineers that have software development background and experience with BI can lead the process of the interface development.
Check our video to learn about data engineering more:
Data engineering, explained
However, business intelligence tools may include a lot of industry-specific functionality. BI developers are also aware of business domain peculiarities: This allows them to understand business requirements, implement proper data models, and data representation solutions. So, the responsibilities of s BI developer are:
- setting business requirements for BI tools;
- translating business requirements into technical ones;
- leading BI software development, deployment, and maintenance;
- report curation and data modeling;
- participation in data warehouse design;
- documenting contents in a data warehouse and meta-data storage; and
- creating technical documentation for BI tools.
Let’s elaborate on each point.
Formulating business requirements for BI tools
Business intelligence is always bound to the actual business domain and requirements of a specific organization to data analysis. That means from company to company BI tools can be totally different. Let’s assume there are two projects:
- A health care organization that works with annual medical examination data. This data is analyzed by a team of domain experts to receive information about state of health in certain region, and across different age groups, genders, social groups, etc. The number of parameters is expanded with diagnostic information, treatment data, and mortality rates. The aim of analytics is to form reports that will compare demographic groups to each other by different factors. Each data group must also be accessible separately to drill down into detailed reviews. The end-user should also be able to create automatic reports monthly.
- An ecommerce platform struggles to improve the work of its customer support. For that purpose, they want to use their website analytics and sales numbers to present reports to customer support agents. Simple data visualization should also be available to view comparisons of traffic and sales during a specific time period.
In both cases, the final product will be a BI platform. But, in the first case, it will require much more advanced options for data analysis, as there are simply more dimensions to be analyzed. The data itself may require image recognition, as it can be given in any format like x-ray, MRI, or ultrasound.
This is the difference in understanding the business requirements for the platform a BI developer should be aware of. Because they will drive the number of reporting facilities, its forms, and additional functionality.
Translating business requirements into technical ones
After the business requirements are gathered and documented, a number of data engineers/ETL/front-end developers are in charge of translating them into technical tasks. A BI developer in this case is responsible for articulating tasks that cover a user interface and reporting capabilities. To be more specific, a BI developer will describe whether a user interface has to be an interactive dashboard with a drag & drop interface, or a set of fill-in fields with data properties, whether a system needs data visualizations and how a user will interact with it as well.
All those decisions will be transformed in the actual tasks for the ETL development team, and BI team (front- and back-end). The development stage can be controlled by a project manager or a BI developer as well.
Leading BI software development, deployment, and maintenance
As mentioned, a BI developer can perform a technical lead role within the development team, or can share the actual developer tasks. But a BI developer can make strategic decisions on the developed software and manage the whole process to its deployment. Testing and debugging can also be a responsibility of a BI developer in the pre’/post-launch stages.
Report curation and data modeling
Reporting is a separate duty that doesn’t concern engineering as much, but rather working with databases and data models. Using a BI interface, users make queries to the database to retrieve some information. Then this information is formatted into a visual form (say, pie chart), or a textual summary by the given properties.
While a user is configuring what to drag and how to format it, a software would be responsible for finding this information and recognizing the required data. This can be done with the help of data modeling — a structure by which the data is stored in the database.
SQL data model
Source: www.mssqltips.com
Data models are defined and applied to the database by data analysts and ETL developers. But, a BI developer is the one who provides reporting requirements, so that will be a determining factor for a future model. These models represent objects in the database and relations between them, so it will determine how data can be queried. And that is part of a BI developer’s competence.
Logically, a BI developer will also control how the data is queried, if it’s valid, and how reporting works.
Participation in a data warehouse design
There are lots of approaches to building unified storage for all corporate information. We discussed these approaches in a dedicated article about warehouses. While the architectural style of a data warehouse doesn’t influence reporting directly, a BI developer may take part in the design process.
Document contents in a data warehouse and meta-data storage
This point goes hand in hand with the previous one. A BI developer can be involved in documenting data structure of a warehouse, as well as meta-data (additional information about data). This documentation will be used during the development process and help with making decisions about the required reporting options.
Create technical documentation for BI interface
Any system needs technical documentation. While a BI developer won’t onboard users to the platform, and show how to use it, he/she’s in charge of providing guidance as a document. This documentation covers the scope of usage for usual users and tech. staff, e.g. API documentation, known technical issues, etc.
The skillset of a BI developer
The BI developer skills can vary depending on the project. In some cases, a BI developer can have minimum interaction with the software engineering and operate as an SQL specialist to ensure proper querying. But for the sake of a full description, we’ll list all the possible skills that fit typical responsibilities.
The list of BI developer’s skills required to execute corresponding responsibilities
Experience with BI tools. First and foremost, every BI developer should have experience developing customer BI products or configuring existing ones. In case of building a custom BI product, Python knowledge, scripting languages, and experience with data visualization libraries are obligatory. Solid understanding of all the underlying infrastructure is also required: This includes the understanding of data integration tools, ETL/ELT processes, data formatting, and warehouse architecture.
If we talk about a product that uses a BI solution by a vendor, then a BI developer has to be acquainted with its corresponding technology stack. These depend on the vendor, but the most popular solutions that concern data pipelines, reporting and database operations are Amazon products (like QuickSight), Hadoop platform, or Apache technologies (like Apache Superset).
DB/DBA background. The main reason why a BI developer should be experienced dealing with databases is that the main subject of concern is SQL/noSQL queries. BI systems require frequently dealing with databases, data models, or distribution of data in several databases.
BI developers can also be involved in designing and building tools for online analytical processing (OLAP cubes). This is an advanced option for multidimensional analysis that utilizes non-relational databases.
Data analysis background. While it doesn’t require a BI developer to have been a data scientist in the past, experience in that field is highly desired. Data modeling and warehouse design are hardly possible without this knowledge, as well as understanding of business requirements for BI systems.
Business analysis skills. The same goes for business analysis. At the start of the project, business analysis in the field of BI will help set precise requirements for future software and make the right strategic decisions.
Software engineering skills. Software developer’s skills depend on the platform you are going to launch your BI interface on. Modern systems are multiplatform, so they can be accessed via mobile devices, through the browser, or as a desktop application. So, knowledge of a programming language depends on the actual platform.
Debugging/troubleshooting. Each stage of designing, developing, and maintaining requires strong troubleshooting skills. Working with data is not that easy. As long as we speak about business data, there is a certain degree of complexity connected with security issues, data access, permissions, updates, and maintenance of the whole platform.
Testing phase is all about finding bugs in the system, but for the most part, a BI developer will fix issues bound to queries and data representation.
When to hire a BI developer?
A perfect scenario for hiring a business intelligence developer is when your requirements for BI software implies more analytical/reporting complexity, rather than data integration and formatting. The BI developer is a specialist that focuses on the user-facing part of the whole data-pipeline. So in terms of efficiency and budgeting, this role is valid in the case of a strong focus on the analytical capabilities. If you consider building a data platform, or some part of it (a warehouse, or data integration tools), then you have to consider hiring the following people:
- an ETL developer,
- a database/warehouse developer,
- a data engineer, and
- a data analyst.
But, to understand what your real focus is for the future BI product, it’s always better to consult with a specialist in the field.