Using Data to Solve Software Pain Points

Photo by Mars on Unsplash

Where to begin

Understanding the Data Landscape

  • Understanding the compliance requirements of data
  • Understanding the data state
  • Understanding who uses the data and how it is used

Understanding Compliance Requirements

Understanding State

Understanding User Needs

  • Are you exposing appropriate endpoints?
  • Are you aggregating data to the user’s needs?
  • Are you naming unnecessary assumptions about how data is used?

Duct tape architecture

Fragility to Reliability

  • Latency: How long do requests take? How many time-outs? Is it your network causing the latency, your dependency calls, or your business logic? Do you have slow database queries you can optimize?
  • Traffic: Are there trends? Are you seeing spikes or anomalies? Do you have backoff and other appropriate re-try practices?
  • Errors: Do you report your errors? Are your errors easy to find, parse, and read? Are your errors handled appropriately? Are your errors actionable?
  • Saturation: Are you saturating for heap/stack/memory? Can you scale up workers? Are you returning appropriate payload sizes? Do you need to batch/paginate your data payloads?




Data Engineer | ML hobbiest | Golang Evangelist | Presenter

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

^Chinese Idiom Stories for Software Professionals: #4 To win a war with words (纸上谈兵)

Cruising Towards a Month

[LeetCode] 99. Recover Binary Search Tree

Watson Speech Services: Introducing a Simple Interface to Train STT and TTS Custom Models!

Expected exception attribute in JUnit 4 is not always the best choice

Understanding Aborters and Cancellation Requests in Web Applications

Open source, How to contribute to open source projects

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Miriah Peterson

Miriah Peterson

Data Engineer | ML hobbiest | Golang Evangelist | Presenter

More from Medium

“How can you make effective decisions if risk cost data is derived by sticking a finger in the air?”

Client numbers matter

Little Data Is Waiting for Its Parents to Pick It Up

Making use of correlations