Jb in a nutshell: Category Cecil

Cecil performance issues

Jb Evain — Tue, 02 Sep 2008 08:45:00 +0200

At the beginning, Cecil was written to be an assembly manipulation library. The initial goal was to be able to read assemblies without loading them in an AppDomain, and also, to expose more that what .net 1.1 provided.

I implemented the writing parts of Cecil during my first Summer of Code, and barely touched it after, at it turns out it was working.

But this summer, two great hackers, Jeroen, author or IKVM, and Rolf, our VB 8 compiler author, decided that they gave enough blood and sweat to work around System.Reflection.Emit issues, and prototyped a version of their project based on Cecil instead.

If Rolf decided to go the full Cecil way, and replaced the usage of System.Reflection and System.Reflection.Emit (his branch is available), Jeroen decided to write a layer on top of Cecil that mimics the System.Reflection.Emit API.

Both wrote a sum-up about their findings, here for Jeroen, and there for Rolf. And both came to the conclusion that Cecil performs a lot less well that System.Reflection.Emit, both in terms of speed and memory consumption.

Rolf also uses the delay loading branch of Cecil, created by Mainsoft for their CIL to Java bytecode translator.

So the point is that Cecil uses too much memory, and is not fast enough in those scenarios, while it performs better in the reading only or reading + manipulating.

The good news is that, thanks to those hackers, we have now two amazing test cases to work on and to optimize. I’ve also started a few weeks ago, as a night hack, a refactoring of Cecil, that removes the intermediate structures that the current version of Cecil uses to read and write assemblies. If that makes reading and writing a little bit more complicated, it should also save a lot of memory.

And who knows, maybe one day, when Cecil will be optimized properly, and that the SRE on top of Cecil layer will be sufficient, it will require only a couple of changes to base mcs on Cecil.

Anyway, kudos to those hackers, let see what I can do to catch up.

We’ll let you know!

Mono.Cecil vs System.Reflection

Jb Evain — Tue, 18 Mar 2008 11:25:00 +0100

My friend Patrick, who’s leading the amazing NDepend effort wrote a great post comparing Cecil against the built-in System.Reflection:

Mono.Cecil vs System.Reflection

Thanks Patrick for the kind words!

Mono.Cecil 0.6

Jb Evain — Fri, 05 Oct 2007 15:42:00 +0200

Yeah, time for a new release of Mono.Cecil.

New since 0.5:

A lot of work to read obfuscated assemblies,
Some precious optimizations,
Improved the Mono.Cecil.Binary layer to read pure native PE binaries,
Ability to annotate Cecil’s objects,
Of course fixed a handful of bugs,
Add a lot or helpers accessors to Cecil’s type, like IsPublic, IsVirtual, Is*,
Preliminary support for debugging symbols (to be blogged about).

Where to download:

Thanks to all the folks that contributed patches to this release!

Beloved users, please upgrade.

Reflexil will rock your IL

Jb Evain — Wed, 19 Sep 2007 19:16:00 +0200

Sébastien Lebreton, friend and Reflexil’s author published a nifty article on codeproject, which explains to the mass how to look clever in society, by inserting wisely chosen opcodes in your methods.

As a matter of fact, this article has first be written in French, and published on the TechHeadBrothers web site.

What’s really brilliant is that Reflexil uses Mono.Cecil to perform the mutations of the assemblies you’ll create.

Clarification

Jb Evain — Fri, 20 Jul 2007 22:33:00 +0200

Eight days ago, I’ve blogged about an issue involving Cecil and ildasm. And the last security update to the .net framework.

After a few hours of investigation, I’ve managed to be able to reproduce the ildasm crash. It turns out that the security update is actually not involved in the issue. It’s just that I stumbled upon right after the security update, and that I had not experienced it before.

So, the ildasm crash happened when I was disassembling an assembly that I had just round-tripped. The issue is that in the same folder as the new assembly, there was the pdb file that was going along the assembly. So when Cecil re-writes the assembly, the methods identifiers are re-generated, and the pdb which keep track of them, becomes out of sync. ildasm uses the pdb file to read the variable names, and other infos. So when it tries to read them from the pdb, which doesn’t match the assembly, it may crash.

You can read the bug reported at Microsoft. It’s somewhat a corner case, so it’s not really critical. But if ever you face the same symptoms while disassembling an assembly generated by Cecil, you can try to delete the pdb.

Mono.Cecil have moved

Jb Evain — Fri, 13 Jul 2007 20:57:00 +0200

Yesterday night was filled with emotion. After more than two years in its own module in SVN, the library Mono.Cecil have moved to a shiner place, right where the core assemblies are, in the mcs module.

Dear Cecil users, you’ll now have to check Cecil out from mcs/class/Mono.Cecil.

Yet, Cecil’s API is not set in stone, so if Cecil makes it to the GAC, you’ll still have to keep a private copy of Cecil for your developments, as it won’t be deployed as a package, you’ll need a private copy during the compilation.

As soon as I can freeze the API, it will be packaged like the other assemblies. But this is another story.

Cecil, ildasm and the security update

Jb Evain — Thu, 12 Jul 2007 18:46:00 +0200

Since the last security update of the .net 2 framework, ildasm may have some issues with assemblies emitted by Cecil. This goes from a pure ildasm crash, to methods that can’t be disassembled. Plus, the behavior is sometimes random. That’s clearly a regression in ildasm, because it used to work, and other disassembler are able to read the assembly (Reflector, monodis).

Still, as it seems to happen only with assemblies emitted by Cecil, Cecil must be doing something that ildasm doesn’t appreciate anymore, but on the other hand, ildasm shouldn’t crash either. I’ve not yet been able to discover what this particular thing is. I’ve submited a bug to Microsoft. Let see if they can help telling what’s going wrong.

Reflexil, easy assembly patching

Jb Evain — Mon, 02 Jul 2007 11:47:00 +0200

My old friend Sébastien pinged me a few days ago because he started to use Cecil. Let simply say that he was modifying some assemblies to circumvent a natural limitation of whatever program.

This long time Delphi user like to have a GUI for this kind of hack, so he simply started to write yet another plugin for Reflector.

The plugin is only in a very early stage, but still, you can deal with primitives types and branches very easily.

You can find here the mandatory introducing post for Reflexil. Even if it’s in french, I bet you’ll locate the binaries as well as the sources easily.

Projects using Cecil

Jb Evain — Mon, 18 Jun 2007 21:41:00 +0200

A month ago, Scott had the good idea to start a page on the Mono.Cecil google group, to try to list the different projects out there that are using Cecil. The page is wisely named Projects using Cecil. I’ve tried to complete it by myself, but I’m for sure missing some of my users.

So if you’re using Cecil, that would be cool to have you listed there. Feel free to change the page yourself, or to drop a name, a summary and if possible a link in the comments.

For bonus points, you can describe what you use Cecil for.

Annotating objects

Jb Evain — Tue, 12 Jun 2007 16:38:00 +0200

Two weeks ago, I’ve borrowed a concept I’ve first seen in Boo to use it in Cecil.

As the title of this entry kindly suggests, it’s about something called annotations. The need for them came from my work on the linker. Before using annotations, to decide which elements to link, I’ve re-created a tree of objects called markers. For instance, to mark a method, I used to have a class similar to this one:

public class MethodMarker : Marker {

    MethodDefinition method;

}

When it was time to sweep methods in a type, I was checking if the method was found in the markers of a type, and if not, I knew I could safely remove it. That worked well, but I was basically re-creating a parallel object tree to the Cecil one, only to support a few more information.

Here came to me the idea of reusing the annotations concept. Let’s have a look at a new interface:

namespace Mono.Cecil {

    using System.Collections;

    public interface IAnnotationProvider {

        IDictionary Annotations { get; }
    }
}

I’ve ensured that every important type like TypeDefinition or MethodDefinition in Cecil implement this type, et voil� , I’m now able to randomly tag them, with whatever data I want.

To keep up with the example of the linker, instead of creating a tree of markers, I now simply annotate a method as «marked», and when it’s time to sweep the unused one, the check is now very simple.

Well, that’s pretty useful to me, hopefully it will be useful to you too.

UPDATE:

Following the excellent suggestion from Bertrand, I’ve changed the way the types implement IAnnotationProvider to an explicit implementation, and it looks really better.