For CTOs looking to squeeze new life out of legacy enterprise applications, the cloud offers tantalizing prospects. Pushbutton scalability, reduced capital costs, outsourcing of non-core IT functions, potentially greater monitoring and health management capabilities and almost certainly greater overall uptime… even with the potential downsides, its no wonder senior management is tempted.
And yet those downsides are more than nagging problems; in many cases they pose significant barriers to a successful move from private data center to public cloud. An existing enterprise app might work fine running on internal hardware, with a modest user base… but move it blindly to a VM in Azure or AWS and suddenly that clever little accounting app grinds to a halt (as does your business). But why is that, exactly?
There are many potential difficulties to overcome when migrating an existing enterprise app to the cloud: reliance on past-generation (read: potentially unsafe) database or file-system drivers that might be ill-suited (or incompatible with) your chosen cloud stack, legal or regulatory requirements that mandate where the data lives, preconceptions about ambient hardware or network infrastructure baked (inadvertently, or otherwise) into your software, security contexts or sandbox privileges required for successful operation that may not be recommended best practices in the cloud, etc. Any of these (and many more) can trip up a migration effort. But there’s one incompatibility so pervasive that its worth discussing further, on its own. Its origins largely pre-date cloud computing itself, in fact. But the negative effects haunt us now, and we’ll likely continue to deal with them for years to come.
It’s your application’s data model.
It’s not the data itself… even if you’ve got a lot of that, there’s plenty of room to store it all in the cloud, if you want. And it’s not the application code per se, though it’s likely that you’ll need to change at least some of that to maximize the full potential of your cloud-hosted application.
No, what I’m talking about is the original conceptual model used to define the database underpinning your application. This model was probably created a long time ago… perhaps you paid a lot of money to a database architect who studied your requirements and used tools like Erwin or ER/Studio to make complex graphical depictions of tables and relationships, or maybe the model was defined by developers in code using APIs like Entity Framework or NHibernate. In either case, you could likely sit down with a developer on your team and have them walk you through the model, and you’d see elements of your business domain that you recognize… a Customer table, defined relationships between an Address and a Warehouse, etc. And this would seem logical and reasonable to you… the application performs some vital function for your business, as part of that it manipulates data, that data needs to live somewhere… voila! Here it is… in the database, created from this model.
The problem is that this model almost certainly has one very big assumption baked into it… it assumes there will be one physical database created from the model, and that all the data will live there. It is by definition a relational model… the concepts modeled within and their relationships (their “referential integrity”) can only be reliably maintained if the data is reasonably co-located, such that the database process can enforce transaction boundaries, data staleness and visibility rules, update query indexes as data changes render them obsolete, process complex joins across multiple tables, etc. Relational databases do not (cannot) reliably do these things across multiple machines. For a more detailed, nerdy explanation of why this is so, see CAP theorem.
In short, a relational model is predicated on the existence of One Giant Database. And unfortunately, sooner or later that single machine is doing as much work as it can do, but you need more. And now you have One Giant Problem.
To be clear, this isn’t really a cloud-specific issue. Any computational- or data-intensive resource (cloud-based or not) will eventually saturate. At that point, you have two options: scale up (buy a bigger server) or scale out (buy more servers). If the resource in question is an application server, either option (assuming your application architect is competent and anticipated scale-out scenarios) can work. But if the resource is a traditional relational database, you really only have two options: scale up and hope it’s good enough, or re-architect for scale-out. Sharding is sometimes a possible third option, sometimes a manifestation of the second… it has it’s place, but also enough drawbacks to make it unsuitable for the general case.
So for relational database scalability issues, scale up is usually the first consideration. More memory, more processors, more and faster disks… these will help your application serve more requests and handle more users, for a time. But you eventually bump up against the laws of physics. There is only so much RAM, CPU, and disk I/O you can bake into a database machine (physical or virtual, cloud-based or not). And even if your data access needs are within reach of the current technological state of the art, they may not be within reach of your budget (a quick perusal of AWS hosted database pricing shows a range from less than 2 cents/hour to over $7.50/hour… cha-ching!).
And so you’re left with the option of re-architecting for scale out. Scale out has two significant advantage over scale up. First, it’s theoretically unbounded; you can keep adding more servers forever. In practice this isn’t even remotely true, as life and software architecture inevitably intrudes and poses some actual upper bound. But still, it’s reasonable to say that a properly architected enterprise application can scale out much, much further than it can ever scale up. The second advantage is cost; scaled out solutions can be done incrementally, and with commodity hardware. This affords you the opportunity to purchase as much scalability as you need at any given moment in time. Scaled up solutions require the use of ever-more-expensive hardware resources, and perhaps worse, necessitate that existing resources must be retired or repurposed… with scale up, you can’t aggregate hardware to work cooperatively (which is exactly what happens in scale out).
But the big disadvantage of scale out is that you have to plan for it, architect for it, and choose technologies that enable it. And there’s the core issue with relational models and scale out; a relational model, and a database created from it, and likely the code written to work with it, are all fundamentally incompatible with any plan to scale out arbitrarily (darn that referential integrity!). Something will have to change, and that something will cost you time and money. There are limited options in products like SQL Server and Oracle for clustering a handful of machines together, but these tend to be used more in service to failover/reliability than pure scalability/availability needs.
So, fine then… relational databases are incompatible with the preferred means of scaling cloud-based software (meaning, scale out). Relational models are poor but frequently used tools for modeling business domains, with significant negative implications for future scalability of the affected applications. But how did this happen? Didn’t we see this coming?
Sure we did. For years, smart people have implored us to stop using (relational) database models as the blueprint for software implementing non-trivial business processes. We just didn’t listen. Our tools (cough Entity Framework cough) make it easy to go from database to code, and while things like EF code first provide us with other modeling alternatives, many applications are still constructed bottom-up from a relational database model. Guess what? If you start with a monolithic relational model and auto-generate EF code to talk to that model, your EF code isn’t any more cloud-ready than your database is (to be clear, I like EF and think it’s entirely appropriate for use on constrained subsets of an otherwise large model, even in the cloud… it’s the naïve use of huge, monolithic EF models that I object to).
“But we’ve always done it this way.” Sure we have. In fairness, that’s not entirely our fault… the skills and tools needed to create a proper domain model independent of a dedicated storage model have for various reasons not yet gained broad traction. The path of least (initial) resistance is to start with a database and build upwards. Legions of enterprise developers have written code like this for years (decades?) and still do. I like to think we’re slowly moving beyond this, and I have high hopes for things like Domain-Driven Design, microservices architectures, and polyglot persistence as some of the practices and patterns that will help us break the cycle. More on that in my next post. But for now, we’re still a long way from industry-wide enlightenment.
We’ve kicked this relational modeling can down the road for a long time, because we could. In a world of small private data centers, modest departmental application needs and manual, Excel-driven business processes, relational databases with relational CRUD-style code on top and built from relational models are not always great but are often good enough. It’s when our ambitions grow, and our anticipated use of these creaky enterprise apps grows along with them, that our best laid plans face the harsh reality of the technical debt we’ve incurred.
You want to move your IT infrastructure to the cloud? You want a more elastic, robust, flexible, agile infrastructure upon which to run your business? That’s a valuable goal. The cloud can give you that, and more. But make plans to retire that technical debt first.
In my next post, I’ll explore ways to do just that… we’ll talk about migration strategies for existing applications, and also touch on ways to minimize that technical debt in the first place.
Until then…
I did an online session today celebrating the 25th anniversary of our Partner Infragistics. During the session, there were a lot of questions I was unable to answer because we ran out of time. Below are those questions and my responses (in italics). Many of these questions are answered by my various WintellectNOW videos. You can register for a free 14-day trial here.
1. Are memory leaks reflected in the used memory statistic of the task manager?
No, Task Manager doesn’t offer the best column for this. Use PerfMon.exe and watch a Process’ Virtual Bytes.
2. As a Windows OS advocate, I am curious on how Threading is implemented in Unix/Linux.
I answered this question on the call.
3. Can a thread ever run (or be set to run) for longer than one quantum?
Can you increase the time quantum for long running tasks
Not directly. You can raise a thread’s priority so it prevents lower-priority threads from running.
4. Can you point out the difference between threads and tasks?
A Task queues an operation to the thread pool. The thread pool then has one of its threads perform the operation. The thread pool threads are re-used over and over again to process all the queued operations. This reduces memory consumption and improves performance.
5. Could you elaborate more on the logical vs. real CPUs?
A physical CPU can perform one operation at a time. But sometimes, the physical CPU must pause and wait for RAM to complete some work. This causes the CPU to sit idle. Hyper-threaded CPUs can execute another thread during these pause times to improve overall system performance.
6. File Open Dialog was a common windows UI Control which exhibits this leak. Are there Windows APIs that tend to "leak" as well?
When you take a dependency on any technology (Microsoft or non-MS), you inherit its performance and efficiency problems. But, you saved yourself some time and energy. As a software developer you are charged with considering this tradeoff and determining if it is worth it for your application and your customers. Also, note that performance and efficiency are moving targets; that is, they change over time with later versions of the technology. With later versions things can get better or worse. So, when you take a dependency on some technology, these are the things you must be thinking about.
7. Given the advantages of using the thread-pool wouldn't it make sense to not allow the developer to create explicit threads and only provide access to "tasks" via the Win API?
Yes, in fact, the Windows Runtime (WinRT) API does not offer any functions allowing you to create threads; you MUST use the thread pool.
8. How are threads from background processes scheduled?
I’m not sure how you exactly define a “background process.” But, for the most part Windows schedules threads in a round-robin fashion without regard to which process the threads are in.
9. How could I make OS to schedule the most of CPU time working for an application?
This is dangerous thing to do and is discouraged. If the app goes into an infinite loop, then the rest of the system suffers greatly. However, you can raise the priority of threads in a process.
10. How does a thread pool help and how many threads should a pool have?
Thread pools help because they create threads and re-use them over and over again. This saves time because they do not constantly create and destroy threads. In addition, the thread pool knows how many CPUs the PC has and tries to create 1 thread per core to reduce context switching; this also improves performance.
11. How is the new Task class implemented? I had heard it was lighter weight than the Thread.
A Task is a small object in memory that knows how to queue a callback method to the thread pool. A task has no threads of its own. The task can monitor the lifetime of the queued item: did it complete? Did it thrown, did it return a value, etc.
12. How to interpret the benefit of parallel computing (increased performance) related to this thread waste of resource?
Most PCs can easily handle the allocation of a few MBs of memory in order for you app to take advantage of parallel processing. If you just strive for no more than 1 CPU per logical core, then memory consumption will stay low and performance will stay high.
13. If a quantum is 30 ms - how long does the context switch take?
How much time does it take the OS to do a context switch (i.e. what is context switch overhead relative to the size of a quantum)?
The time for a context switch various based on many factors: CPU speed, CPU architecture, and so on. But, what makes the performance even worse is that the CPUs cache is usually invalid after a context switch causing a lot of cache misses when accessing RAM.
14. If it is beneficial to create less thread as possible then what strategy should we choose when creating a WPF responsive Application? Any pattern or approach?
Typically UI apps (like WPF) have 1 GUI thread that process all user-interface events. Then, you queue up computationally-intensive work to the thread pool (via a Task) allowing the UI thread to respond to user input.
15. Jeff, even today we see some situations where we are not able to bring up task manager when the system is extremely busy.. why is this happening
It’s hard for to know for sure without being in front of the machine. But, my guess is that there may be some high-priority threads that are preventing Task Manager from displaying. This can sometimes happen with a bad device driver too.
16. Question set about duration of quantum, which you've indicated is 30ms - Has it always been 30 ms? What governs this duration? The HAL/clock interval?
Yes, the PC’s clock interval. There is a Win32 function that returns this info: GetSystemTimeAdjustment. Look at the lpTimeIncrement return value.
17. So are threads automatically recycled after a certain period of not being used?
If the thread are thread pol threads, then yes. If they are not thread pol threads, then no.
18. Threading topics apply to Azure programming?
Yes. Azure just creates virtual machines with Windows or other operating systems running in them. So information about threads apply to these VMs as well.
19. ThreadPool.GetAvailableThreads() show the answer as 1024 instead of 8. How is that possible when the number of cores in my PC are just 8?
In the remarks section for this method, it says that it returns the “number of additional worker threads that can be started.” This says that the thread can create this menay threads but not that it has actually created them.
20. What happens if you open another File Save As? Will more threads be orphaned?
No, it reuses the threads it created previously.
21. What is difference between Foreground and Background thread.
The .NET CLR kills a process as soon as all its foreground threads have stopped running (instantly destroying any background threads). So, foreground threads keep a process running while background threads do not.
22. What is fiber support in case of threading
Fibers a light-weight threads with the OS kernel doesn’t know anything about. The developer must write code to “context switch” from one fiber to another. Each fiber does have its own user mode stack but all fibers (on the same thread) share a kernel mode stack. .NET doesn’t support fibers and probably never will.
23. What is the performance effect of having multiple CPUs with multiple cores vs a single CPU with an equivalent number of cores?
For the most part, the perf would be the same. Sometimes, CPUs have to communicate with each other (like when taking a thread synchronization lock) and this communication is faster if the CPUs don’t have to talk through the bus on the motherboard.
24. How does async/await relate to threading?
These C# and VB language features allow you to perform I/O operations without blocking threads. This reduces the number of threads an application/service needs decreasing resource consumption and improving performance. I explain the value of all this in the video available here.
25. Is it always 30ms? Even in Windows 8.1?
On all versions of Windows to date, the timer interrupt fires at the same rate.
26. How do we get the additional threads started by File Open Dialogs (and other such controls) returned back to Thread Pool ?
You can’t control what a component does.
27. So those logical processors have their own CPU cache too? Otherwise, won't that impact performance?
The logical processors typically share the CPU’s cache. This is usually good as it allows bytes to be read once into the cache and shared by the other logical processors.
28. Is it wasteful to switch contexts if all the threads on the system are waiting most of the time?
No. But, if a thread does not want to wait, then allowing it to run without context switching is faster than introducing context switching.
29. Why is 32-bit arm about 1/2 of 32-bit x86 for overhead for memory for thread kernel object? Less data? More packing?
ARM CPUs have fewer CPU registers.
30. My question was: will there ever be such a concept supported by the .NET runtime, like the BEAM (Erlang virtual machine) threads, which said to be far more efficient and less resource expensive than threads, quicker context switch.
I’m not familiar with BEAM threads and what this actually means. I assume that they are like fibers and it is very unlikely that .NET will ever support fibers.
