In high-performance software engineering, it’s easy to become so focused on optimization that you lose sight of broader business needs. This article shares lessons from my 20+ years building real-time, high-load systems across finance, telecom, and advertising. I explain how engineering intuition—shaped by edge-case systems—can become a liability when applied blindly. The goal: help engineering teams, managers, and even non-technical leaders understand when performance matters, and when it doesn’t.
The world of high-load, low-latency systems is a crucible of engineering disciplines. It distills software down to its most brutal essence: cycles, bytes, wire time. It teaches you to think like a CPU, to treat L1 cache behavior as gospel, and to view allocations with suspicion. It sharpens your instincts for system limits and exposes any architectural indecision under fire at 200,000 RPS. Like all specializations, it also distorts your lens.
In this piece, I want to reflect on the cost of repeatedly optimizing for edge cases. Specifically, how the deeply internalized mindset from building ultra-performant, latency-sensitive systems can become a limiting factor when engineers work in more elastic, less deterministic business domains.
Where Intuition Begins
I started my journey in real-time systems, where timing guarantees are absolute. Like brakes in a car, the system simply cannot be late. Over time, I transitioned into web systems where elasticity and geo-distribution took precedence over millisecond determinism. In banking, consistency and security overtook performance. Then, in adtech, I encountered the most complex compromise yet: soft real-time expectations (under 100ms), low cost per request, and architectural elasticity, all under high traffic conditions.
This varied experience taught me that an engineer must think vertically, understanding the entire system from business goals to infrastructure. However, it also exposed the trap many fall into: carrying over instincts honed in one domain into another, without adapting them.
Latency as Reflex
In high-load systems with latency constraints, your thinking changes. You focus not on modularity or reusability, but on minimizing processing. The logic is data-centric: don’t transform the data. Shape your logic to fit the data. Don’t pass structures between microservices. Instead, keep them where they are and compute directly. Don’t copy—compute in place.
These habits are powerful. But the engineer must also understand the variability of input data, the business flow that justifies it, and the degree of correctness required. And they must know when such reflexes start working against the business.
Pathological Optimization in General Systems
An engineer who grew up working on high-load systems often assumes the bottleneck is always latency. But most systems are not bound by that constraint. Over the years, I’ve seen engineers apply data-centric programming, sharded databases and even custom databases, to CRUD backends. The performance gain was marginal; the complexity cost was vast.
One example: while working at MTS and AdNow, we optimized the DSP backend to hit 200k RPS. The temptation was to optimize everything. We learned to isolate the high-performance core and treat admin panels, user dashboards, and analytics with simpler tools like Postgres and standard libraries. Not every component needed to live at the edge.
When Experience Becomes Bias
Engineers carry the weight of their successes. For example, I avoided sharding if it was possible to do the processing on a single server, albeit with a significantly expanded RAM volume (for example, up to 4TB), and this often gave excellent results both in the cost of maintenance and in development – it is much easier to write code when all the data is in your memory than to work with sharded data. In such cases, we scaled vertically, adding RAM.
However, this same intuition can become counterproductive in systems where horizontal scale is natural and cheap. Intuition must be regularly retrained. A 10-core server may outperform a 40-core box unless NUMA topology and mutex contention are addressed. You must consider switches (Cut-Through vs. Store-and-Forward), storage behavior (95th percentile latency), and data access patterns.
The mature engineer’s mindset is not “What is the best possible system?” but “What does the business need, and how can we get there with minimum future pain?”
Debuggability vs. Efficiency
Another anti-pattern from high-performance thinking: sacrificing observability for CPU savings. In one system, we replaced JSON logging with binary formats to reduce overhead. It worked—3% CPU saved. But the trade-off? Incident response slowed, human debugging suffered, and onboarding became a challenge.
In ultra-constrained systems like SCADA (which I worked on early in my career), this trade-off is acceptable. But in most cloud-native systems, human time trumps compute time. Engineers should always ask: “Who’s going to debug this at 3 a.m.?”
Cultural Drift in Engineering Teams
I’ve seen high-performance specialists bring over habits that don’t scale organizationally: rewriting libraries, resisting abstractions, and insisting on building from scratch. I’ve been guilty of it too, creating “hot” functions in x86-64 assembler and using directly the processor’s SIMD commands (AVX-512), even when using the Go language. These systems work brilliantly, but only in the hands of their authors. By the way, assembler is very conveniently integrated into Go, but, as practice has shown, few people know it.
Today, as a manager, I emphasize that business tasks must come first. If business requires creating a system with extreme performance characteristics, then we must remember that this will require a stellar team and vice versa, if a standard system is required, then it is not worth using solutions that are difficult to support and develop, but should focus on schemes and architecture that are understandable to the mass developer, which can be mastered by a mid-level developer in two weeks.
I also caution against microservices sprawl. Teams eager to optimize development workflows may inadvertently turn a tight data-centric pipeline into a network of chatty, latency-laden services. Even Amazon Prime Video teams have returned from microservices to monoliths in such contexts.
Unlearning the Habit of Optimization
The hardest thing for performance-oriented engineers to learn is when not to optimize. When building the MTS platform, we had parts of the system that were hot paths and others that were barely used. Initially, the urge was to use the same tooling across all components. But it’s a trap. CRUD services don’t need to scale like your bidding engine. Choose the right tool, even if that means sacrificing uniformity.
Similarly, early in my career, when developing code to perform a fast inverse Fourier transform, I wrote a highly optimized assembler code that outperformed GCC by 30%. But six months later, the processors changed, and my code optimized for prefetch pipelines began to perform worse than the compiler-generated one. That’s the real cost of low-level perfection: it rarely lasts. Performance tuning should be reserved for business-critical paths.
Lifecycle Cost of Optimization
What often gets overlooked is the total lifecycle cost of early optimization. An efficient algorithm or exotic data layout may save compute time today, but if it slows down onboarding, complicates testability, or becomes a barrier to architectural evolution, then the system is effectively accruing technical debt disguised as technical brilliance. I’ve seen projects where an engineer’s initial low-level perfection became a tax the team paid for years—refactoring was too risky, and new hires avoided touching critical modules. Mature systems require performance that evolves with the business, not just code that’s clever in the moment. The question we should always ask is: “Can this optimization survive team growth and domain shifts over 3–5 years?”
Learning to Let Go
In our CTR/VTR predictor, we send updates in real time. But for the segment coverage calculator in the UI, we switched to batch processing. The latency of updating the data in the UI has increased, but the complexity of support and development has decreased significantly without any impact on the business goal.
The wisdom is not in knowing how to optimize, but in knowing when the business doesn’t need it. Premature optimization is only “evil” when it’s misaligned with business maturity. If you’re targeting 200k RPS on a core, yes—start sharp. But if it’s 200k RPS on a cluster, and you can shard data processing (e.g. distribute requests across nodes), then act iteratively, start with inefficient but working code, and plan refactoring to achieve lower server load. And how to use the “extra” hardware freed up after optimization has never been a problem.
Mature Engineers Understand Trade-offs
The most balanced engineers I’ve worked with understand the entire vertical, from business requirements to infrastructure tuning. They know when to use Apache Ignite and when Postgres is enough. They make performance decisions only where bottlenecks align with business outcomes.
Such engineers avoid optimization addiction. They favor maintainability. They are patient with temporary solutions and confident that the roadmap allows for future improvements. That confidence, along with technical range, is what I consider “technical maturity.”
Leading Self-Organizing Teams Without Overengineering
Left unchecked, high-performance teams often over-abstract. To prevent that, I encourage cross-functional ownership—engineers who think like analysts, testers, and product managers. I advocate for teams that talk openly, document and justify the architectural decisions (ADR), and include domain experts in the loop. You cannot succeed with brilliant engineers alone. You need people from the market.
Also, I’ve seen over-specialization derail early-stage teams. For new systems, buy experienced teams or foundational platforms. Don’t reinvent everything. Starting from scratch without market-ready expertise often leads to failure, unless backed by large investments.
Final Thoughts
I don’t regret specializing in high-load systems. It shaped how I think. But I’ve also seen how these instincts, left unchecked, can limit engineers in broader domains. We must relearn general software engineering just as athletes must relearn walking after years of sprinting.
Your best skill can become your biggest bias. And your greatest strength, when balanced, is what makes you a truly versatile engineer.
If you’re a product leader, founder, or manager: don’t ask your engineers to over-optimize from the start. Let them build clear, maintainable systems that can scale later if needed. High performance has its place—but business alignment always comes first.
