Difference: FRCVLabMLSA_description (1 vs. 8)

Revision 82017-07-12 - AnneMarieBogar

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />
Changed:
<
<

Architecture

>
>

MLSA_logo.png Architecture

  Lightweight program (which we call filters) operate on program source files and/or data files and produce data files. The filters can be stacked in pipelines, where each filter in the pipeline reads data files generated by prior filters and in turn generates new data files. The design motivation behind this structure is to allow pipelines of filter programs to be constructed to implement program analysis. This modular design is important to isolate the language-specific first pipeline stages from later language-independent modules and in this way support sophisticated analysis for multilingual condebases.
Line: 43 to 42
  -- (c) Fordham University Robotics and Computer Visio
Deleted:
<
<
META FILEATTACHMENT attachment="callgraph.png" attr="" comment="" date="1469630804" name="callgraph.png" path="callgraph.png" size="40117" user="DamianLyons" version="1"
 
META FILEATTACHMENT attachment="C_AST.jpeg" attr="" comment="" date="1469630804" name="C_AST.jpeg" path="C_AST.jpeg" size="36767" user="DamianLyons" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="PY_AST.jpeg" attr="" comment="" date="1469630804" name="PY_AST.jpeg" path="PY_AST.jpeg" size="19537" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="process_diagram.png" attr="" comment="" date="1469631588" name="process_diagram.png" path="process_diagram.png" size="44954" user="DamianLyons" version="1"
 
META FILEATTACHMENT attachment="pythonJSON.png" attr="" comment="python AST" date="1491593154" name="pythonJSON.png" path="pythonJSON.png" size="21514" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="jsJSON.png" attr="" comment="javascript AST" date="1491593251" name="jsJSON.png" path="jsJSON.png" size="26339" user="DamianLyons" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="call_graph.png" attr="" comment="" date="1491593790" name="call_graph.png" path="call_graph.png" size="211415" user="DamianLyons" version="1"
 
META FILEATTACHMENT attachment="Call_Graph_diagram.png" attr="" comment="" date="1491924500" name="Call_Graph_diagram.png" path="Call Graph diagram.png" size="42121" user="MaggieGates" version="1"
META FILEATTACHMENT attachment="test6_callgraph.png" attr="" comment="" date="1499805913" name="test6_callgraph.png" path="test6_callgraph.png" size="440537" user="AnneMarieBogar" version="1"
META FILEATTACHMENT attachment="Static_Analysis_diagram.png" attr="" comment="" date="1499870065" name="Static_Analysis_diagram.png" path="Static_Analysis_diagram.png" size="31113" user="AnneMarieBogar" version="1"
Added:
>
>
META FILEATTACHMENT attachment="MLSA_logo.png" attr="" comment="" date="1499887540" name="MLSA_logo.png" path="MLSA_logo.png" size="39428" user="AnneMarieBogar" version="1"

Revision 72017-07-12 - AnneMarieBogar

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />
Changed:
<
<

How does it work?

>
>

Architecture

Lightweight program (which we call filters) operate on program source files and/or data files and produce data files. The filters can be stacked in pipelines, where each filter in the pipeline reads data files generated by prior filters and in turn generates new data files. The design motivation behind this structure is to allow pipelines of filter programs to be constructed to implement program analysis. This modular design is important to isolate the language-specific first pipeline stages from later language-independent modules and in this way support sophisticated analysis for multilingual condebases.

  Static_Analysis_diagram.png
Line: 26 to 28
  Figure 3: Portion of a JavaScript AST json file
Changed:
<
<
The AST files have very different structures for C, Python and Javascript, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files, detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future. The version can also handle PyV8 's eval function to call a JavaScript program from Python. In the future, the program will be able to handle cases in which a JavaScript program is called from a C program, and both JavaScript and Python functions are called from C programs.
>
>
The AST files have very different structures for C, Python and Javascript, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files, detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis has been implemented for C/C++ programs that call Python programs (but none of the other languages) to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future. The version can also handle PyV8 's eval function to call a JavaScript program from Python, and JQuerry's ajaz function to call a Python program from JavaScript. In the future, the program will be able to handle cases in which a JavaScript program is called from a C program, and both JavaScript and Python functions call C programs.
 
Changed:
<
<

When the designated function used to call another program of another language, such as “PyRun_SimpleFile” or JSContext().eval(), is found, its argument (name of the Python or JavaScript file) is considered a function call and the executable portion of that file is represented as the main function in the original program. That creates the connection between the two files, which allows the subsequent programs to build the call graph.

CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph via GraphViz and saves it as a PDF file. Circular nodes represent C programs, rectangular nodes represent Python programs, and hexagonal nodes represent JavaScript programs. Recursive functions are denoted by dotted nodes.

>
>

When the designated function used to call another program of another language, such as “PyRun_SimpleFile” , JSContext().eval() or $.ajax(), is found, its argument (name of the Python or JavaScript file) is considered a function call and the executable portion of that file is represented as the main function in the original program. That creates the connection between the two files, which allows the subsequent programs to build the call graph.

“mergeFunCall.py” combines all individual csv files from the list of source files into one. This file is then used as input to “generateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph via GraphViz and saves it as a PDF file. Circular nodes represent C programs, rectangular nodes represent Python programs, and hexagonal nodes represent JavaScript programs. Recursive functions are denoted by dashed nodes. Errors, such as circularity in a system or unidentifiable interoperability, are denoted by double-lined dashed nodes.

  test6_callgraph.png

Revision 62017-07-12 - AnneMarieBogar

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />

How does it work?

Changed:
<
<
Static_Analysis_diagram.png
>
>
Static_Analysis_diagram.png
 
Changed:
<
<
Call_Graph_diagram.png
>
>
Call_Graph_diagram.png
  The process starts with C, Python and JavaScript source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)
Line: 30 to 30
 

When the designated function used to call another program of another language, such as “PyRun_SimpleFile” or JSContext().eval(), is found, its argument (name of the Python or JavaScript file) is considered a function call and the executable portion of that file is represented as the main function in the original program. That creates the connection between the two files, which allows the subsequent programs to build the call graph.

CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph via GraphViz and saves it as a PDF file. Circular nodes represent C programs, rectangular nodes represent Python programs, and hexagonal nodes represent JavaScript programs. Recursive functions are denoted by dotted nodes.

Changed:
<
<
test6_callgraph.png
>
>
test6_callgraph.png
  Figure 4: Example of a multilingual call graph
Line: 49 to 49
 
META FILEATTACHMENT attachment="jsJSON.png" attr="" comment="javascript AST" date="1491593251" name="jsJSON.png" path="jsJSON.png" size="26339" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="call_graph.png" attr="" comment="" date="1491593790" name="call_graph.png" path="call_graph.png" size="211415" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="Call_Graph_diagram.png" attr="" comment="" date="1491924500" name="Call_Graph_diagram.png" path="Call Graph diagram.png" size="42121" user="MaggieGates" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="Static_Analysis_diagram.png" attr="" comment="" date="1491924512" name="Static_Analysis_diagram.png" path="Static Analysis diagram.png" size="144072" user="MaggieGates" version="1"
 
META FILEATTACHMENT attachment="test6_callgraph.png" attr="" comment="" date="1499805913" name="test6_callgraph.png" path="test6_callgraph.png" size="440537" user="AnneMarieBogar" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Static_Analysis_diagram.png" attr="" comment="" date="1499870065" name="Static_Analysis_diagram.png" path="Static_Analysis_diagram.png" size="31113" user="AnneMarieBogar" version="1"

Revision 52017-07-11 - AnneMarieBogar

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />
Line: 30 to 30
 

When the designated function used to call another program of another language, such as “PyRun_SimpleFile” or JSContext().eval(), is found, its argument (name of the Python or JavaScript file) is considered a function call and the executable portion of that file is represented as the main function in the original program. That creates the connection between the two files, which allows the subsequent programs to build the call graph.

CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph via GraphViz and saves it as a PDF file. Circular nodes represent C programs, rectangular nodes represent Python programs, and hexagonal nodes represent JavaScript programs. Recursive functions are denoted by dotted nodes.

Changed:
<
<
call_graph.png
>
>
test6_callgraph.png
  Figure 4: Example of a multilingual call graph
Line: 50 to 50
 
META FILEATTACHMENT attachment="call_graph.png" attr="" comment="" date="1491593790" name="call_graph.png" path="call_graph.png" size="211415" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="Call_Graph_diagram.png" attr="" comment="" date="1491924500" name="Call_Graph_diagram.png" path="Call Graph diagram.png" size="42121" user="MaggieGates" version="1"
META FILEATTACHMENT attachment="Static_Analysis_diagram.png" attr="" comment="" date="1491924512" name="Static_Analysis_diagram.png" path="Static Analysis diagram.png" size="144072" user="MaggieGates" version="1"
Added:
>
>
META FILEATTACHMENT attachment="test6_callgraph.png" attr="" comment="" date="1499805913" name="test6_callgraph.png" path="test6_callgraph.png" size="440537" user="AnneMarieBogar" version="1"

Revision 42017-04-11 - MaggieGates

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />

How does it work?

Changed:
<
<
process_diagram.png
>
>
Static_Analysis_diagram.png

Call_Graph_diagram.png

  The process starts with C, Python and JavaScript source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)
Line: 12 to 14
  Figure 1: Portion of a C AST file
Changed:
<
<
the AST module and the file ast2json.py (for Python files),
>
>
the AST module and the file ast2json.py (for Python files),
  pythonJSON.png

Figure 2: Portion of a Python AST json file

Changed:
<
<
And SpiderMonkey for JavaScript files.
>
>
And SpiderMonkey for JavaScript files.
  jsJSON.png

Figure 3: Portion of a JavaScript AST json file

Changed:
<
<
The AST files have very different structures for C and Python, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future.

When a call to “PyRun_SimpleFile” is found, its argument (name of the python file) is considered a function call and the executable portion of that python file is represented as the main function in C. That creates the connection between the C and the Python files, which allows the subsequent programs to build the call graph.

>
>
The AST files have very different structures for C, Python and Javascript, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files, detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future. The version can also handle PyV8's eval function to call a JavaScript program from Python. In the future, the program will be able to handle cases in which a JavaScript program is called from a C program, and both JavaScript and Python functions are called from C programs.
 
Changed:
<
<
“CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph and saves it as a PDF file. Circular nodes represent C functions and Rectangular nodes represent Python functions.
>
>

When the designated function used to call another program of another language, such as “PyRun_SimpleFile” or JSContext().eval(), is found, its argument (name of the Python or JavaScript file) is considered a function call and the executable portion of that file is represented as the main function in the original program. That creates the connection between the two files, which allows the subsequent programs to build the call graph.

CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph via GraphViz and saves it as a PDF file. Circular nodes represent C programs, rectangular nodes represent Python programs, and hexagonal nodes represent JavaScript programs. Recursive functions are denoted by dotted nodes.

  call_graph.png
Line: 48 to 48
 
META FILEATTACHMENT attachment="pythonJSON.png" attr="" comment="python AST" date="1491593154" name="pythonJSON.png" path="pythonJSON.png" size="21514" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="jsJSON.png" attr="" comment="javascript AST" date="1491593251" name="jsJSON.png" path="jsJSON.png" size="26339" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="call_graph.png" attr="" comment="" date="1491593790" name="call_graph.png" path="call_graph.png" size="211415" user="DamianLyons" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Call_Graph_diagram.png" attr="" comment="" date="1491924500" name="Call_Graph_diagram.png" path="Call Graph diagram.png" size="42121" user="MaggieGates" version="1"
META FILEATTACHMENT attachment="Static_Analysis_diagram.png" attr="" comment="" date="1491924512" name="Static_Analysis_diagram.png" path="Static Analysis diagram.png" size="144072" user="MaggieGates" version="1"

Revision 32017-04-07 - DamianLyons

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />

How does it work?

Changed:
<
<
process_diagram.png
>
>
process_diagram.png
 
Changed:
<
<
The process starts with C and Python source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)
>
>
The process starts with C, Python and JavaScript source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)
 
Changed:
<
<
C_AST.jpeg
>
>
C_AST.jpeg
  Figure 1: Portion of a C AST file
Changed:
<
<
and the AST module (for Python files).
>
>
the AST module and the file ast2json.py (for Python files),
 
Changed:
<
<
PY_AST.jpeg
>
>
pythonJSON.png
 
Changed:
<
<
Figure 2: Portion of a Python AST file
>
>
Figure 2: Portion of a Python AST json file
 
Changed:
<
<
The AST files have very different structures for C and Python, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future.
>
>
And SpiderMonkey for JavaScript files.
 
Changed:
<
<
When a call to “PyRun_SimpleFile” is found, its argument (name of the python file) is considered a function call and the executable portion of that python file is represented as the main function in C. That creates the connection between the C and the Python files, which allows the subsequent programs to build the call graph.
>
>
jsJSON.png
 
Changed:
<
<
“CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph and saves it as a PDF file. Circular nodes represent C functions and Rectangular nodes represent Python functions.
>
>
Figure 3: Portion of a JavaScript AST json file
 
Added:
>
>
The AST files have very different structures for C and Python, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future.
 
Changed:
<
<
callgraph.png
>
>
When a call to “PyRun_SimpleFile” is found, its argument (name of the python file) is considered a function call and the executable portion of that python file is represented as the main function in C. That creates the connection between the C and the Python files, which allows the subsequent programs to build the call graph.
 
Added:
>
>
“CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph and saves it as a PDF file. Circular nodes represent C functions and Rectangular nodes represent Python functions.
 
Changed:
<
<
Figure 3: Example of a multilingual call graph
>
>
call_graph.png
 
Added:
>
>
Figure 4: Example of a multilingual call graph
 

Permissions

Line: 43 to 45
 
META FILEATTACHMENT attachment="C_AST.jpeg" attr="" comment="" date="1469630804" name="C_AST.jpeg" path="C_AST.jpeg" size="36767" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="PY_AST.jpeg" attr="" comment="" date="1469630804" name="PY_AST.jpeg" path="PY_AST.jpeg" size="19537" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="process_diagram.png" attr="" comment="" date="1469631588" name="process_diagram.png" path="process_diagram.png" size="44954" user="DamianLyons" version="1"
Added:
>
>
META FILEATTACHMENT attachment="pythonJSON.png" attr="" comment="python AST" date="1491593154" name="pythonJSON.png" path="pythonJSON.png" size="21514" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="jsJSON.png" attr="" comment="javascript AST" date="1491593251" name="jsJSON.png" path="jsJSON.png" size="26339" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="call_graph.png" attr="" comment="" date="1491593790" name="call_graph.png" path="call_graph.png" size="211415" user="DamianLyons" version="1"

Revision 22016-07-27 - BrunoVieiro

Line: 1 to 1
 
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />
Changed:
<
<

Process Description

>
>

How does it work?

 
Changed:
<
<
process_diagram.png
>
>
process_diagram.png
  The process starts with C and Python source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)

C_AST.jpeg

Changed:
<
<
Figure 2: Portion of a C AST file
>
>
Figure 1: Portion of a C AST file
  and the AST module (for Python files).

PY_AST.jpeg

Changed:
<
<
Figure 3: Portion of a Python AST file
>
>
Figure 2: Portion of a Python AST file
  The AST files have very different structures for C and Python, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future.
Line: 28 to 28
 callgraph.png
Changed:
<
<
Figure 4: Example of a multilingual call graph
>
>
Figure 3: Example of a multilingual call graph
 

Permissions

Revision 12016-07-27 - DamianLyons

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="FRCVLabMLSA"
<meta name="robots" content="noindex" />

Process Description

process_diagram.png

The process starts with C and Python source code from which separate ASTs (Abstract Syntax Tree) are dumped using Clang-Check (for C files)

C_AST.jpeg

Figure 2: Portion of a C AST file

and the AST module (for Python files).

PY_AST.jpeg

Figure 3: Portion of a Python AST file

The AST files have very different structures for C and Python, but the parsers are designed to handle each kind of AST differently. Those parsers filter the AST files detecting and recording function calls and their arguments. Initially, the program is capable of detecting literals and variables as arguments. Reaching Definition Analysis will be developed in the future in order to handle statically assigned variables as arguments to functions. The current version of the program handles part of the Python.h interface between C and Python. It only analyzes “PyRun_SimpleFile” calls. Other mechanisms for calling Python from C will also be implemented in the future.

When a call to “PyRun_SimpleFile” is found, its argument (name of the python file) is considered a function call and the executable portion of that python file is represented as the main function in C. That creates the connection between the C and the Python files, which allows the subsequent programs to build the call graph.

“CombineCSV.py” combines all individual csv files from the list of source files into one. This file is then used as input to “GenerateDot.py”. This program translates the csv file to a dot file, which represents the csv file as a graph. The Dot program builds the final graph and saves it as a PDF file. Circular nodes represent C functions and Rectangular nodes represent Python functions.

callgraph.png

Figure 4: Example of a multilingual call graph

Permissions

  • Persons/group who can view/change the page:
  • Set ALLOWTOPICCHANGE = FRCVRoboticsGroup

-- (c) Fordham University Robotics and Computer Visio

META FILEATTACHMENT attachment="callgraph.png" attr="" comment="" date="1469630804" name="callgraph.png" path="callgraph.png" size="40117" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="C_AST.jpeg" attr="" comment="" date="1469630804" name="C_AST.jpeg" path="C_AST.jpeg" size="36767" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="PY_AST.jpeg" attr="" comment="" date="1469630804" name="PY_AST.jpeg" path="PY_AST.jpeg" size="19537" user="DamianLyons" version="1"
META FILEATTACHMENT attachment="process_diagram.png" attr="" comment="" date="1469631588" name="process_diagram.png" path="process_diagram.png" size="44954" user="DamianLyons" version="1"
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback