Your browser may have trouble rendering this page. See supported browsers for more information.

This page shows the source for this entry, with WebCore formatting language tags and attributes highlighted.

Title

Iterating with NDepend to remove cyclic dependencies (Part II)

Description

In the previous article, we discussed the task of <a href="{app}view_article.php?id=430">Splitting up assemblies in Quino using NDepend.</a> In this article, I'll discuss both the high-level and low-level workflows I used with NDepend to efficiently clear up these cycles. Please note that what follows is a description of how I have used the tool---so far---to get my very specific tasks accomplished. If you're looking to solve other problems or want to solve the same problems more efficiently, you should take a look at the <a href="http://www.ndepend.com/docs/getting-started-with-ndepend">official NDepend documentation</a>. <h>What were we doing?</h> To recap briefly: we are reducing dependencies among top-level namespaces in two large assemblies, in order to be able to split them up into multiple assemblies. The resulting assemblies will have dependencies on each other, but the idea is to make at least <i>some</i> parts of the Encodo/Quino libraries opt-in. <h>The plan of attack</h> On a high-level, I tackled the task in the following loosely defined phases. <dl dt_class="field"> Remove direct, root-level dependencies This is the big first step---to get rid of the little black boxes. I made NDepend show only direct dependencies at first, to reduce clutter. More on specific techniques below. Remove indirect dependencies <div><img attachment="encodo_namespace_black_hole.png" align="right" caption="Direct and Indirect references (the Black Hole)">Crank up the magnification to show indirect dependencies as well. This will will help you root out the remaining cycles, which can be trickier if you're not showing enough detail. On the contrary, if you turn on indirect dependencies too soon, you'll be overwhelmed by darkness (see the depressing initial state of the Encodo assembly to the right).</div> Examine dependencies between root-level namespaces <div>Even once you've gotten rid of all <i>cycles</i>, you may still have unwanted dependencies that hinder splitting namespaces into the desired constellation of assemblies. For example, the plan is to split all logging and message-recording into an assembly called <c>Encodo.Logging</c>. However, the <c>IRecorder</c> interface (with a single method, <c>Log()</c>) is used practically <i>everywhere</i>. It quickly becomes necessary to split interfaces and implementation---with many more potential dependencies---into two assemblies for some very central interfaces and support classes. In this specific case, I moved <c>IRecorder</c> to <c>Encodo.Core</c>. Even after you've conquered the black hole, you might still have quite a bit of work to do. Never fear, though: NDepend is there to help root out those dependencies as well.</div> Examine cycles in non-root namespaces <div>Because we can split off smaller assemblies regardless, these dependencies are less important to clean up for our current purposes. However, once this code is packed into its own assembly, its namespaces become root namespaces of their own and---voila! you have more potentially nasty dependencies to deal with. Granted, the problem is less severe because you're dealing with a logically smaller component. In Quino, use non-root namespaces more for organization and less for defining components. Still, cycles are cycles and they're worth examining and at least plucking the low-hanging fruit.</div> </dl> <h>Removing root-level namespace cycles</h> With the high-level plan described above in hand, I repeated the following steps for the many dependencies I had to untangle. Don't despair if it looks like your library has a ton of unwanted dependencies. If you're smart about the ones you untangle first, you can make excellent---and, most importantly, <i>rewarding</i>---progress relatively quickly.<fn> <ol> Show the dependency matrix Choose the same assembly in the row and column Choose a square that's black Click the name of the namespace in the column to show sub-namespaces Do the same in a row Keep zooming until you can see where there are dependencies that you don't want Refactor/compile/run NDepend analysis to show changes <c>GOTO 1</c> </ol> <h>Once again, with pictures!</h> The high-level plan of attack sounded interesting, but might have left you cold with its abstraction. Then there was the promise of detail with a focus on root-level namespaces, but alas, you might still be left wondering just how <i>exactly</i> do you reduce these much-hated cycles? I took some screenshots as I worked on Quino, to document my process and point out parts of NDepend I thought were eminently helpful. <h>Show only namespaces</h> <img attachment="show_namespaces_involved_in_dependency.png" align="left" caption="Show Namespaces involved in a Dependency"><img attachment="encodo_namespace_cycles_namespace_reference_only.png" align="right" caption="Reference cycles in Encodo (Namespaces only)">I mentioned above that you should <iq>[k]eep zooming in</iq>, but how do you do that? A good first step is to zoom all the way out and show only direct <i>namespace</i> dependencies. This focuses only on <c>using</c> references instead of the much-more frequent member accesses. In addition, I changed the default setting to show dependencies in only one direction---when a column references a row (blue), but not vice versa (green). As you can see, the diagrams are considerably less busy than the one shown above. Here, we can see a few black spots that indicate cycles, but it's not so many as to be overwhelming.<fn> You can hover over the offending squares to show more detail in a popup. <clear><h>Show members</h> <img attachment="encodo_namespace_cycles.png" align="left" caption="Reference cycles in Encodo (Members)"><img attachment="bind_matrix_headers_to_cluster_cycles.png" align="right" caption="Bind Matrix to Cluster Cycles Together">If you don't see any more cycles between namespaces, switch the detail level to "Members". Another very useful feature is to "Bind Matrix", which forces the columns and rows to be shown in the same order and concentrates the cycles in a smaller area of the matrix. As you can see in the diagram, NDepend then highlights the offending area and you can even click the upper-left corner to focus the matrix only on that particular cycle. <clear><h>Drill down to classes</h> <img attachment="show_classes_involved_in_dependency.png" align="left" caption="Show Classes involved in a Dependency"><img attachment="dependency_arrows.png" align="right" caption="Use the handy arrow dependency-indicators">Once you're looking at members, it isn't enough to know just the namespaces involved---you need to know which types are referencing which types. The powerful matrix view lets you drill down through namespaces to show classes as well. If your classes are large---another no-no, but one thing at a time---then you can drill down to show which <i>method</i> is calling which method to create the cycle. In the screenshot to the right, you can see where I had to do just that in order to finally figure out what was going on. In that screenshot, you can also see something that I only discovered after using the tool for a while: the direction of usage is indicated with an arrow. You can turn off the tooltips---which are informative, but can be distracting for this task---and you don't have to remember which color (blue or green) corresponds to which direction of usage. <h>Indirect dependencies</h> <img attachment="encodo_namespace_black_hole_smaller.png" align="left" caption="The black hole is almost gone"><img attachment="encodo_namespace_members_only_smaller.png" align="right" caption="No more black squares and a 5-element cycle left">Once you've drilled your way down from namespaces-only to showing member dependencies, to focusing on classes, and even members, your diagram should be shaping up quite well. On the right, you'll see a diagram of all direct dependencies for the remaining area with a problem. You don't see any black boxes, which means that all <i>direct</i> dependencies are gone. So we have to turn up the power of our microscope further to show indirect dependencies. On the left, you can see that the scary, scary black hole from the start of our journey has been whittled down to a small, black spot. And that's with all direct and indirect dependencies as well as both directions of usage turned on (i.e. the green boxes are back). This picture is much more pleasing, no? <h>Queries and graphs</h> <img attachment="ndepend_query_language.png" align="left" caption="NDepend Query Language"><img attachment="view_graph_to_see_cycle.png" align="right" caption="View graph to see cycle from Culture to Enums (through Expression)"><img attachment="zoom_on_culture_and_enums_and_indirect.png" align="right-column" caption="Show indirect (zoom) in on enums & culture">For the last cluster of indirect dependencies shown above, I had to unpack another feature: NDepend queries: you can select any element and run a query to show using/used by assemblies/namespaces.<fn> The results are shown in a panel, where you can edit the query see live updates immediately. Even with a highly zoomed-in view on the cycle, I still couldn't see the problem, so I took NDepend's suggestion and generated a graph of the final indirect dependency between <c>Culture</c> and <c>Enums</c> (through <c>Expression</c>). At this zoom level, the graph becomes more useful (for me) and illuminates problems that remain muddy in the matrix (see right). <clear><h>Crossing the finish line</h> In order to finish the job efficiently, here are a handful of miscellaneous tips that are useful, but didn't fit into the guide above. <img attachment="encodo_assembly_is_finally_clean.png" align="right" caption="Encodo assembly is finally clean"><ul> I set NDepend to automatically re-run an analysis on a successful build. The matrix updates automatically to reflect changes from the last analysis and won't lose your place. If you have ReSharper, you'll generally be able to tell whether you've fixed the dependencies because the usings will be grayed out in the offending file. You can make several fixes at once before rebuilding and rerunning the analysis. At higher zoom levels (e.g. having drilled down to methods), it is useful to toggle display of row dependencies back on because the dependency issue is only clear when you see the one green box in a sea of blue. Though Matrix Binding is useful for localizing, remember to toggle it off when you want to drill down in the row independently of the namespace selected in the column. </ul> And BOOM! just like that<fn>, phase 1 (root namespaces) for Encodo was complete! Now, on to Quino.dll... <h>Conclusion</h> <img attachment="refactoring_82087_symbols.png" align="left" caption="Refactoring 82,087 Symbols...">Depending on what shape your library is in, <i>do not</i> underestimate the work involved. Even with NDepend riding shotgun and barking out the course like a rally navigator, you still have to actually make the changes. That means lots of refactoring, lots of building, lots of analysis, lots of running tests and lots of reviews of at-times quite-sweeping changes to your code base. The destination is worth the journey, but do not embark on it lightly---and don't forget to bring the right tools.<fn> <hr> <ft>This can be a bit distracting: you might get struck trying to figure out which of all these offenders to fix <i>first</i>.</ft> <ft>I'm also happy to report that my initial forays into <i>maintaining</i> a relatively clean library---as opposed to <i>cleaning</i> it---with NDepend have been quite efficient.</ft> <ft>And much more: I don't think I've even scratched the surface of the analysis and reporting capabilities offered by this ability to directly query the dependency data.</ft> <ft>I'm just kidding. It was a lot of time-consuming work.</ft> <ft>In this case, in case it's not clear: NDepend for analysis and good ol' ReSharper for refactoring. And ReSharper's new(ish) architecture view is also quite good, though not even <i>close</i> to detailed enough to replace NDepend: it shows assembly-level dependencies only.</ft>