srcML

srcML
Developer(s)	Michael L. Collard Jonathan I. Maletic
Written in	C, C++, Java
License	GPL
Website	www.srcml.org

srcML (source M L) is a document-oriented XML representation of source code. It was created in a collaborative effort between Michael L. Collard and Jonathan I. Maletic. The abbreviation, srcML, is short for Source Markup Language. srcML wraps source code (text) with information from the Abstract Syntax Tree or AST (tags) into a single XML document. All original text is preserved so that the original source code document can be recreated from the srcML markup. The only exception is the possibility of newline normalization.^[1]

The purpose of srcML is to provide full access to the source code at the lexical, documentary, structural, and syntactic levels. The format also provides easy support for fact-extraction and transformation. It is supported by the srcML toolkit maintained on the srcML website and has been shown to perform scalable, lightweight fact-extraction and transformation.

srcML toolkit

The srcML toolkit consists of the command-line program called srcml, which translates from source code to srcML when provided a code file on the command line or translates from srcML to source code when a srcml archive is provided on the command line. The program also supports direct queries and transformations of srcML archives using tools like XPath, XSLT, and RELAXNG. The srcML toolkit is actively maintained and currently support C, C++, C#, and Java.

srcML format

The srcML format consists of all text from the original source code file plus XML tags. Specifically, the text is wrapped with srcML elements that indicate the syntactic structure of the code. In short, this explicitly identifies all syntactic structures in the code.^[2]

The tags used in srcML are listed out below along with what category they fall within.

Category	srcML Elements
File/Project	unit
Statement	asm, block, break, case, comment, condition, constraint, continue, decl, decl_stmt, default, do else empty_stmt, enum, expr, expr_stmt, extern, for, goto, if, incr, index, init, label, macro, name, namespace, range, requires, switch, template, then, type, typedef, using, while
Function/Method	argument, argument_list, call, function, function_decl, param, parameter_list, return, specifier
Class	class, class_decl, constructor, constructor_decl, destructor, destructor_decl, friend, member_list, private, protected, public, super
Struct and Union	struct, struct_decl, union, union_decl
Exception	catch, throw, throws, try
C-Preprocessor	cpp:define, cpp:directive, cpp:elif, cpp:endif, cpp:error, cpp:file, cpp:if, cpp:ifdef, cpp:ifndef, cpp:include, cpp:line, cpp:pragma, cpp:undef
K&R C	krparam, krparameter_list
Java	extends, finally, implements, import, package
Extra Markup	lit:literal, type:modifier, op:operator
Debug	marker, mode, parse
Misc	escape

srcML uses XML namespaces. Below is a list of the prefix used to denote each namespace, and the namespaces themselves.

Prefix	Namespace
src	http://www.srcml.org/srcML/src
cpp	http://www.srcml.org/srcML/cpp
pos	http://www.srcml.org/srcML/position
lit	http://www.srcml.org/srcML/literal
err	http://www.srcml.org/srcML/srcerr

Note: for a srcML archive, the entire project will be contained within a single root unit element, and each individual file will be contained as a unit element within the root unit element.

Single file conversion

The following shows how srcml can be used on single files.

The following example converts the C++ file main.cpp to the srcML file main.cpp.xml:

srcml main.cpp -o main.cpp.xml

The following command will extract the source code from the file main.cpp.xml and place it into the C++ file main.cpp:

srcml main.cpp.xml -o main.cpp

Project conversion

The following shows how src2srcml and srcml2src can be used with an entire project:

The following example converts the project 'project' to the srcML file project.xml

srcml project -o project.xml

The following command will extract the source code files from the file project.xml and place it into the directory project:

srcml —to-dir project project.xml

Program transformation with srcML

srcML allows the use of most if not all current XML APIs and tools to write transformations. It also allows for the use of XSLT directly using the argument—xslt={name}.xls on the srcml2src command. Using srcML's markup with XSLT allows the user to apply Program Transformations on an XML-like structure(srcML) to obtain transformed xml which can then be written back its source code representation using the srcml2src tool. The application of srcML to program transformation is explained, in detail, by Collard et al.^[1]

The following command will run the XSLT program program.xsl on the srcML archive project.xml

srcml —xslt program.xsl project.xml

Fact extraction with srcML

In it simplest form, Fact Extraction using srcML leverages XPath in order to address parts of the srcML document and pull information about various entities or characteristics of the source code. Of course, it is not limited to this. Any standard XML API may be used. The application of srcML to fact extraction is explained, in detail, by Kagdi et al.^[3]

cpp:directive, cpp:file, cpp:include, cpp:define, cpp:undef, cpp:line, cpp:if, cpp:ifdef, cpp:ifndef, cpp:else, cpp:elif, cpp:endif, cpp:then, cpp:pragma, cpp:errorliteral, operator, modifier An example to create a srcML archive from an entire software project.

The following command runs the XPath path on a srcML archive project.xml

srcml —xpath "xpath" project.xml

Work is being done on providing convenient extension functions.

Source code difference analysis with srcML

srcML brings a lot of advantages to doing difference analysis on source code. One of these advantages is the ability to query for differences between specific sections of a codebase as well as across versions of the same codebase. The application of srcML for difference Analysis is explained, in detail, by Maletic et al.^[4]

Examples

As an example of how srcML is used, here is an XPath expression that could be used to find all classes in a source document:

//src:class

Another example might be finding all comments within functions:

/src:function//src:comment

Due to the fact that srcML is based on xml, all XML tools can be used with srcML, which provides rich functionality.

References

^ ^a ^b Collard, M.L., Decker, M., Maletic, J. I. (2011). Lightweight Transformation and Fact Extraction with the srcML Toolkit. Proceedings of the 11th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'11). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)
^ Collard, M.L, Maletic, J. I., Robinson, B.P. (2010). A Lightweight Transformational Approach to Support Large Scale Adaptive Changes. Proceedings of the IEEE 26th IEEE International Conference on Software Maintenance (ICSM'10). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)
^ Collard, Michael L., Kagdi, Huzefa, Maletic, Jonathan I. (2003). An XML-Based Lightweight C++ Fact Extractor. Proceedings of the 11th IEEE International Workshop on Program Comprehension (IWPC'03). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)
^ Maletic, Jonathan I., Collard, Michael L. (2004). Supporting Source Code Difference Analysis. Proceedings of the IEEE International Conference on Software Maintenance (ICSM'04). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

Maletic, Jonathan I., Collard, Michael L., Kagdi, Huzefa (2004). Leveraging XML Technologies in Developing Program Analysis Tools. Proceedings of the 4th International Workshop on Adoption-Centric Software Engineering (ACSE'04). pp. 6 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)
Collard, Michael L., Maletic, Jonathan I., Marcus, Andrian (2002). Supporting Document and Data Views of Source Code. Proceedings of the 2nd ACM Symposium on Document Engineering (DocEng'02). pp. 8 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)
Maletic, Jonathan I., Collard, Michael L., Marcus, Andrian (2002). Source Code Files as Structured Documents. Proceedings of the 10th IEEE International Workshop on Program Comprehension (IWPC'02). pp. 4 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

External links

Official website

[SCAM11-1] Collard, M.L., Decker, M., Maletic, J. I. (2011). Lightweight Transformation and Fact Extraction with the srcML Toolkit. Proceedings of the 11th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'11). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

[ICSM10-2] Collard, M.L, Maletic, J. I., Robinson, B.P. (2010). A Lightweight Transformational Approach to Support Large Scale Adaptive Changes. Proceedings of the IEEE 26th IEEE International Conference on Software Maintenance (ICSM'10). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

[IWPC03-3] Collard, Michael L., Kagdi, Huzefa, Maletic, Jonathan I. (2003). An XML-Based Lightweight C++ Fact Extractor. Proceedings of the 11th IEEE International Workshop on Program Comprehension (IWPC'03). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

[ICSM04-4] Maletic, Jonathan I., Collard, Michael L. (2004). Supporting Source Code Difference Analysis. Proceedings of the IEEE International Conference on Software Maintenance (ICSM'04). pp. 10 pages.{{cite book}}: CS1 maint: multiple names: authors list (link)

[1]

[2]

[3]

[4]