SDK:Writing automated tests: Difference between revisions

From Vectorworks Developer
Jump to navigation Jump to search
(Created page with "{{LocationMain|category=LocationSDKSpecial|specific=}} <div class="rightmenu"> __TOC__ </div> By [mailto:vstanev@nemetschek.net Vladislav Stanev]")
 
No edit summary
Line 4: Line 4:
</div>
</div>


By [mailto:vstanev@nemetschek.net Vladislav Stanev]
By [mailto:swilcox@nemetschek.net Scot Wilcox]
 
==Automated Testing==
It's possible to automate testing your plugin.  This will speed up your development process, since you can execute your test whenever you make a change and it will only take a few seconds.
 
In order to write an automated test, put the following code into a source file in your plugin:
<code lang="cpp">
VW_TEST_CASE(YourTestName, your.email@domain.net, VectorWorks::Debug::ESystemTestType::SpecialRun)
{
    TEST_OpenDocument vwDoc("YourTestFile.vwx");
    CATCH_REQUIRE(vwDoc.IsOpened());
    //Perform your operations, this is a dummy example
    MCObjectHandle layer = gSDK->GetActiveLayer(); 
    //Assert that your operation was performed successfully
    CATCH_REQUIRE(layer != nullptr);
}
</code>
and then put this code into your ModuleMain.cpp:
<code lang="cpp">
    REGISTER_CATCH_SYSTEMTESTS(action, moduleInfo, iid, inOutInterface, cbp, reply);
</code>
 
TEST_OpenDocument is a class that opens a test file and then closes it when the variable goes out of scope.
 
CATCH_REQUIRE checks an expression for true or false.  Use this to check that your code works the way you intended it to.  If it fails, then both sides of the expression with the line and file name will be printed to the output.
 
==Running Automated Tests==
When you run Vectorworks, open the Debug menu and select "System Test Manager."  Look for your test name, then click to the left of it to bring up a check mark.  Then click the "Run" button, and the output will show up in the
"Output" field.  If you have a test file, put the directory the test file is in into the "Test File Directory" field. 
 
You can also run automated tests from the command line with these arguments:
 
-t YourTestName -ab -tf /path/to/your/test/directory -l path/to/your/test/output.txt
 
You can pass in multiple tests like this:
 
-t Test1 -t Test2
 
You can also pass in a text file with test names on new lines:
 
-t testList.txt 
 
==Testing best practices==
Isolate a piece of functionality.  It’s better to call the function you’re interested in directly rather than trying to simulate menu actions or button clicks.  If you can eliminate unnecessary variables, your test won’t fail for other reasons besides what you’re intending to test.
 
Think carefully about what this function is supposed to do.  Don’t assert things that aren’t a direct product of the function getting called.  We don’t want to assert side effects or things that are incidental to the function’s purpose. This is one of the main benefits of testing: it forces you to think about your design and nail down exactly how everything should work.
 
Try to break it.  After you make sure it works in all the normal use cases, if you have time you should throw weird edge cases in there too.  Assume that someone is going to use this function completely wrong.  Does it fail cleanly, or does it crash Vectorworks?
 
Test the most frequently used code paths.  What part of your code is going to get used most frequently by your customers?  Focus your testing time on that.
 
Check your assumptions.  What assumptions does the code you’re testing make?  Are you completely sure that those assumptions are always valid?  If you test that assumption in your automated test, if there’s ever a time when that assumption doesn’t work anymore, the test will notify you.

Revision as of 18:35, 2 January 2019

.SDK|SDK ..SDK:Types|SDK Types ..SDK:Using the SDK|Using the SDK ..VCOM:VCOM (Vectorworks Component Object Model)|VCOM Basics ..VCOM:Class Reference|VCOM Class Reference

By Scot Wilcox

Automated Testing

It's possible to automate testing your plugin. This will speed up your development process, since you can execute your test whenever you make a change and it will only take a few seconds.

In order to write an automated test, put the following code into a source file in your plugin:

VW_TEST_CASE(YourTestName, your.email@domain.net, VectorWorks::Debug::ESystemTestType::SpecialRun)
{
    TEST_OpenDocument vwDoc("YourTestFile.vwx");
    CATCH_REQUIRE(vwDoc.IsOpened());
    //Perform your operations, this is a dummy example
    MCObjectHandle layer = gSDK->GetActiveLayer();   
    //Assert that your operation was performed successfully
    CATCH_REQUIRE(layer != nullptr);
}

and then put this code into your ModuleMain.cpp:

    REGISTER_CATCH_SYSTEMTESTS(action, moduleInfo, iid, inOutInterface, cbp, reply);

TEST_OpenDocument is a class that opens a test file and then closes it when the variable goes out of scope.

CATCH_REQUIRE checks an expression for true or false. Use this to check that your code works the way you intended it to. If it fails, then both sides of the expression with the line and file name will be printed to the output.

Running Automated Tests

When you run Vectorworks, open the Debug menu and select "System Test Manager." Look for your test name, then click to the left of it to bring up a check mark. Then click the "Run" button, and the output will show up in the "Output" field. If you have a test file, put the directory the test file is in into the "Test File Directory" field.

You can also run automated tests from the command line with these arguments:

-t YourTestName -ab -tf /path/to/your/test/directory -l path/to/your/test/output.txt

You can pass in multiple tests like this:

-t Test1 -t Test2

You can also pass in a text file with test names on new lines:

-t testList.txt

Testing best practices

Isolate a piece of functionality. It’s better to call the function you’re interested in directly rather than trying to simulate menu actions or button clicks. If you can eliminate unnecessary variables, your test won’t fail for other reasons besides what you’re intending to test.

Think carefully about what this function is supposed to do. Don’t assert things that aren’t a direct product of the function getting called. We don’t want to assert side effects or things that are incidental to the function’s purpose. This is one of the main benefits of testing: it forces you to think about your design and nail down exactly how everything should work.

Try to break it. After you make sure it works in all the normal use cases, if you have time you should throw weird edge cases in there too. Assume that someone is going to use this function completely wrong. Does it fail cleanly, or does it crash Vectorworks?

Test the most frequently used code paths. What part of your code is going to get used most frequently by your customers? Focus your testing time on that.

Check your assumptions. What assumptions does the code you’re testing make? Are you completely sure that those assumptions are always valid? If you test that assumption in your automated test, if there’s ever a time when that assumption doesn’t work anymore, the test will notify you.