| The Ribosome Builder Project |
| A software project to simulate the ribosome |
|
|
|
Found crashing bug while mapping a sequence from E. coli to 1fka.A in release version: Runtime error R6025: pure virtual function call.
This occurred running the following script: 'script run C:\PROJ\RBO\lua\SeqMap_16S_Ecoli_1fka.lua'
In updating the makefiles in the project, I tried substituting the hardcoded path separator with a macro, but MS Tools are so brain-dead that they get too confused in parsing it ("\\"). Will just have to hardcode the backslash separator for all paths.
Finished changes in makefiles, can now build relative to the src directory. Also added the option to specify a different build path, but that is not tested yet. Also added the option to specify a different install path, but also not tested.
The intent of the reorganization is to be able to define a module in CVS, called 'src' that can be checked out, and then the user can cd into the src dir, type 'nmake /f makefile.win' and build from there. Also, all recursive makefiles are designed to build from their local path, independently of a recursive call.
A second intent is to have the 'src' module be independent of the 'pdb' and 'exp' modules. The 'src' module contains all the files needed to build, install and run a working core application.
Then, there can be a higher-level makefile at the rbuilder dir level, which can do a (build/install/dist) and incrementally use the 'src', 'pdb', and 'exp' modules, as found.
After a little more thought, I will rename 'src' to 'bas'. The 'bas' module contains code for the basic functionality of the program. This seems less confusing than 'src', because other modules such as 'exp' can contain source code (lua scripting code) as well. Also, 'bas' can be thought of as a complement to 'exp' (expansion, experiments, etc.).
Did first submission of code to sf CVS: imported 'bas' module. Did test checkout, build, install and run of bas module on ribo4. It works.
Checked out bas module on beryllium (Mandrake Linux 8.1, Kernel 2.4.8, gcc ver. 2.96). Created first working Makefile for lib/qhull.
Created linux porting progress table for project home page.
Did bug fix and quick release of ver 0.3.1.
Added 'LogHistory=TMAR' to rbuilder CVSROOT/config file, but there is no history file and couldn't find any documentation of what all the flag meanings are for 'TOEFWUPCGMAR'. I guess I will just have to keep track of tags in my own files for now.
Imported 'exp' module from 'exp-0.1.13.zip'. Arg, forget to add 'exe' to cvswrappers! The exe files in exp/tools simply failed to go in. I'll just leave them out of the CVS module for now.
Imported 'pdb' module from 'pdb-0.1.13.zip'.
In doing further development of the Win32 version on ribo4, I'm experimenting with moving the working copies of 'exp' and 'pdb' INTO the 'bas' working directory, so that they will function as up-to-date checked out versions that are used by a running version of the program that has been installed into the 'bas' directory itself. I will create a .cvsignore file in 'bas' to ignore the 'exp' and 'pdb' directories, as well as other files copied to the 'bas' directory during the install. I don't think this should cause any problems. It allows me to have separate modules at the top level in the CVS repository that can be checked out independently, and also located in a suitable nested manner on the local host. I just have to remember to go into the 'exp' and 'pdb' directories explicitly to do any needed updates. I have seen references to doing things with nested modules and aliases with the 'modules' file in CVSROOT, but I don't see a need to do that at this time.
Fixed path separators in VC60 project workspace file in bas/app/rbq/mak. Added VC60 project files to build htsrv and dependency libs (now name 'mmlibs' to distinguish them from the app-specific libs rbqUtil.lib, rbqLua.lib, frameWk.lib). 'frameWk' should really be renamed 'rbqFramewk'.
Also added an rbq_Install VC60 project, so the entire thing can now be built, installed and run from the IDE, eliminating the old batchfile install step.
Added ssh keys and figured out how to get ssh authentication for cvs working on GNU/Linux. It was nearly all straightforward:
Tested VC IDE full build of program on inga, found that rbq.opt file is also needed. I considered adding it to CVS, but it is a binary file and has undesired hardcoded paths in it, typical of poor tool design. Instead, I'll just post text instructions on the build.html page to add the Executable and Working Dir to the rbq_Install settings. Done.
Create scripts listFuncs.pl and utReport.pl in app/rbq/tst/rbqlua. HTML report of progress of unit test definition added to web page docs.
Fixed bug in project makefiles: made sure /F switch preceded all makefile.win files because new linux Makefile broke qhull build.
Added intra-doc hrefs for rbq_XXX lua funcs in app/rbq/tools/doctools/rbq_lua_doc.pl script.
Created first rbqlua unit test app/rbq/tst/rbqlua/funcs/rbq_InsPdbFile.
Redesigned TestUtil.lua functions to better support test cases. Added test driver to run all rbq-lua unit tests.
Added source and include folders for low-level libs in mmlibs project in VC IDE project workspace.
Added support to glob directories from filespec in wkFileUtils::GetFilenames() and rbq_GetFile(), rbq_GetTotalFiles().
Added second test case to rbqlua unit test rbq_InsPdbFile, which called the function with a missing parameter, after disabling lua_error(), and it worked. The error was reported to app stdout, but termination of lua execution was bypassed, allowing the script to continue running.
Next: write ut for rbq_OpenPdbFile() and add code to bypass 'save current doc' dialog.
An interesting press release at Syrrx (http://www.syrrx.com) leads me to think about the possibility that x-ray crystallography could be accelerated past the current bottleneck of trying to get molecules to form good crystals. Syrrx received a grant for a proposal to develope nanoscale structural frameworks that may provide better conditions for the formation of hard-to-crystallize molecules (such as membrane proteins). Should this prove successful, one can envision a whole new field devoted to the design of such structural frameworks, where a framework is customized to support the crystallization of a particular target molecule. More info on the Syrrx grant is located in the project brief at NIST
After noticing a bug in the movie code, and doing a fix, tried to run the mRNA wrap demo and found that the program is not picking up script run request files in the input directory after they are submitted there by htsrv. Found the bug: because of the change to 'wkFileUtil::GetFilenames()', the '.' directory is now being returned. I was going to make the default exclude arg to GetFilenames() be the A_SUBDIR, but I changed my mind for some reason. Oh, it was because I didn't want to include <io.h> in the wkFileUtils.h header file.
After looking, it seems like the input dir filescan is the only other place in the code that uses GetFilenames(), so I'll just put the exclude subdirs param there. Fixed. Movie export also works.
In addition to the rbqlua unit tests, should probably create tests to check browser interface and run demos.
Created another unit test: rbq_GetPdbName(), added more cases to previous tests. Created another unit test: rbq_GetPdbIndexFromName().
possible order of next unit test definitions:
rbq_GetPdbFileName(), rbq_GetObjectIndexFromName(),
rbq_GetTotalChainsInPdb(), chain data access functions,
res access
atom access
object access
selection ops
geom ops:
rbq_GetPdbBoxCenter(), rbq_GetPdbMassCenter(), rbq_GetPdbTransformMatrix()
Started 'app objects' page in whitepaper.
Created a menu item to run a sub-set of unit tests (so the user doesn't have to keep running through the entire list each time, just to verify a few new tests).
Created unit test: rbq_GetChainName().
Created unit test: rbq_GetTotalAtomsInResidue().
next: rbq_GetTotalInterResidueBonds().
Created guide for writing unit tests.
In looking through the code, this particular function appears only to be used in the following rbq_XXX lua api functions:
Next, rbq_GetTotalInterResidueBonds() is called also called just once by its wrapper getTotalInterResidueBonds() in _PdbUtil.lua. Finally, getTotalInterResidueBonds() does not appear to be called by any other lua code. Which is strange.
Perhaps I was just trying to make it a general-purpose function. If so, it needs to be redefined to avoid the ambiguity. Done. Added swap code in CRbqPdbMol::GetInterResBondIndexList( iChain, iRes, jRes ), and some additional specifications in the rbq_XXX functions.
All current 18 rluts run and pass.
Changes were made in the following scripts:
Changes in the corresponding func getAtomIndexFromFullName() => getAtomIndex():
The corresponding function rbq_GetAtomName(), and its PdbUtil.lua wrapper getAtomName(), are also misnamed because they actually return just the PDB atom name, with spaces. This is inconsistent with rbq_GetPdbName(), rbq_GetChainName(), and rbq_GetResidue(). Might as well rename this function now, to rbq_GetAtomPdbName(), and create a new function rbq_GetAtomName(), with the consistent behavior.
After a little more thought, the new name should be 'getAtomParName()' where 'Par' stands for 'PDB ATOM record', as this allows for an associate function 'getAtomPartName()', which returns the PDB ATOM record 'trimmed' name. Done.
Fixed doxygen target in top-level makefile in bas module.
Added independent c-o-m calculations to unit test rbq_GetPdbMassCenter()
Finally learned to use the '-d' option with update in cvs to get new directories.
Noticed a drawing bug where selecting the background in the graphics window causes part of chain 1tra.a to be drawn with a light brown color.
Added another test case to rbq_GetTotalForceObjectsInDynamicSegment.
Read some of the project development pages on the Python home page. Great background info for how to do collective project development.
Interesting reading of Will Ware's nanocad mailing list back in 1995.
Created rlut rbq_GetElementMass.
More reading of the nanocad mailing list (http://discuss.foresight.org/~pcm/nanocad). Starts off with discussions about goals of nanocad and some initial work on energy minimization and how to represent the data, using Lisp. Then around Feb 1996, a lot of talk about how to extend the core chem computational part with a scripting language and gui and what language to use (Java, Python, Tk/Tcl, etc.), and the reasons why (elegance, portability, efficiency, availability, capability, etc). Very interesting to me as it leads me to think of my own choice of C++ and Lua, and the benefits/drawbacks of that. Konrad Hinsen appears to have exceptionally good insight into both the chemistry and programming end of things, and is a proponent of Python. The subsequent success of PyMOL provides confirmation of his inclinations.
Investigating MMTK.
Created rlut rbq_RotateForceObject.
Adding retrieval of node children and parent info to rbq_GetForceObjProperty().
Created rlut rbq_GetObjectIndexFromName.
While doing some debugging involving rotation of a point around a rotation center through multiplication by a homogeneous transform matrix, I browsed a little of the CGrTransform3f code and how its implemented and used by CRbqNodeBsjForce in timestep movement and one blatant inefficiency is the code
// update group atoms pos
for( int iAtom = 0; iAtom < pNode->GetTotalAtoms(); ++iAtom )
{
CTorAtom *pAtom = pNode->GetAtom( iAtom );
CGrVector3f txPos = pAtom->GetUnmovedPos() * txFrame;
pAtom->SetPos( txPos );
}
Because this results in a huge number of unnecessary operations being repeating for each atom inside of CGrTransform3f::TransformVector() where GetMatrix() is called:
//---------------------------------------------------------------------------------------------
// GetMatrix - float
//---------------------------------------------------------------------------------------------
void CGrTransformT::GetMatrix( float rgd[16] ) const
{
// produce the hmTr = hmR * hmT
float hmR[16];
m_mRot.GetMatrix( hmR );
float hmT[16];
SetIdentityMatrixf( hmT );
hmT[12] = m_vTrans[0];
hmT[13] = m_vTrans[1];
hmT[14] = m_vTrans[2];
MultiplyMatrixOpenGLStyle( hmR, hmT, rgd );
}// GetMatrix - float
As soon as I finish the unit tests and can start doing some profiling, this may be one of the bottlenecks.
Added rbo test module to rbuilder exp module as 'tst' subdir. Got exp/tst/testGrtRot.lua working, after writing multiplyVectorAndHMatrix() in VectorUtil.lua. Hopefully this will put a nail in my ever-recurring rotation matrix confusions.
In rlut rbq_RotateForceObjectNode(), calculation of expected node positions using VectorUtil functions is now matching rotation produced by CRbqNodeBsjForceObject, with the proviso that the degree argument to getGrtRotation() is negated.
Added CBnd::GetAtomIndexPairs() to return a flat list of bonds defined by pairs of indices from the packed multilist representation. Also starting documenting bond representation design in the whitepaper docs.
In trying to get baseline data on intra-res bonds before adding property to rbq_GetForceObjProperty(), tried to run lua/ForceObjectReport.lua and found that it was broken in my dev copy of the bas module because the perl and gnuplot subdirs were not added to cvs (need to update the cvswrappers file in CVSROOT so they can be added).
TODO: update blog with split-brain realization.
Debugging rotation in rlut, it seems that a positive rotation fails for node 1 of res 0, and a negative rotation passes for res 0 and fails for node 0 of res 1. Strange.
In the other case, the CRbqNodeBsjForceObject::RotateAboutNode() function is called. In this case, the vector about which rotation occurs is determined by CTorNode::GetDir(). That explains the problem. In CTorNode::GetDir(), the direction is defined from the atom in the parent node to the atom in the node. We don't want to change that definition, because of all the possible problems in forcefield calculations that may result, so the one in CRbqDynamicSegment::RotateNode() will be changed, the one for doing rotation of node 0.
Done. Rbq-Lua unit test rbq_RotateForceObjectNode() is fully passing now.
Created index page for web/wp.
Straightened out documentation of linkage status of adjacent residues during forward and back node rotation.
Added testing of back-rotation of force object nodes. The rfon rlut is finally passing with full testing of nodes and force objects in both foward and back node rotation.
Started getting ready to do rluts for graphics functions. Created command 'dev dumpGraphicsWin' to output current graphics buffer to a bmp file. However, will also need to add additional support functions to adjust the main window and graphics window sizes to a specific size so that the graphics dumps will match a reference file.
Downloaded multiple papers on forcefields for whitepaper references.
Trying to get unit test rbq_DrawBitmapText to work. The problem is that after resizing the main window to a fixed known size, and then doing a write of the graphics window buffer, the graphics child window has not been resized. After a call to resize the main window, the CRbqGraphicsWindow::resizeGL() method is not being called.
Created some documentation for the main window and child widget layouts while working on the graphics window resize problem. Been reading through the Qt interest mail archives, but no enlightenment yet. I think the problem is that the status window and toolbars are rapidly turned off and then shortly afterwards, turned back on again, so perhaps some messages are being combined in an event message queue to prevent the graphics winodw from being resized as desired. The way to figure it out will probably involve overriding some event methods in the Qt classes and doing some printfs during the sequence.
Before doing that, I think I may see the cause of the problem: all of these hide() and show() events are happening sequentially inside of a single script function, and the re-layout/ resize() events that must occur in Qt will naturally happen after the script function has completed. It seems that I'm wanting some 'immediate', non-queued calls of the re-layout/resize and code.
After adding a resizeEvent() override in CRbqGraphicsWindow, the above hypothesis has been confirmed.
By defining an optional 'tickTest.lua' script within a unit test directory, a unit test can be executed by a script object test driver, using timer ticks in the application main event loop. This will allow a script to initiate operations and then check for the expected application behavior in subsequent blocks of execution code, after a particular block of script code has completed. This will solve the graphics window resize problem above, as well as allow for testing of forcefield behavior across multiple timesteps, which is not possible in the current testing scheme. Will also create associated drivers, 'runAllTickTests.lua' and 'runSubsetTickTests.lua'.
Added documentation of main window architecture to wp.
Created first draft of wp outline. Created first draft of goals doc in wp. Reading (MacKerell, 1995).
Creating some more screenshots.
Note: nice sky bgcolor: .64, .65, .88
Trying to fix resizeMainWin() to set size of frame instead of client area (not as easy as it should be because apparently in X11, the frame decoration can be inconsistent, so Qt is weak in this area).
By having working move and resize functions for the window frame, this will save a lot of time because it allows scripted setup of the window position for working with the browser interface, instead of the continual manual resizing and positioning that I have been doing up until this point. Move and resize funcs are working. It is good.
Working on 'lua/CircleObject.lua' script, it is currently drunk. I think I will rewrite it and do it a little differently, without the pitching. Just some simple turns, up/down and forward/back moves.
Actually, simplified it even further by chucking the initial code that sets the view while moving to the circle radius. New version of 'CircleObject.lua' is working.
Starting on creation of tick unit tests. This requires creation of some scripted way of deleting script objects.
Implementing new command 'script so sdelObj' (schedule delete script object). In doing so, I reviewed my earlier log entry about script object deletion (projectLog_2000-2004.html, 2004-02-24), which gives some further explanation of why deletion of script objects from within script object executing code must currently be prohibited (one reason is to prevent document closing, which causes deletion of all script objects and other annotations). At present, there will be some duplicated code with regard to deleting script objects and deleting annotations, but no time to worry about that right now.
Created test script 'exp/dev/tst/testDeleteSo.lua'. Scheduled deletion of script objects appears to be working.
Finally added a helper function for creating script objects, 'constructScriptObject()', to _ScriptUtil.lua.
Initial version of app/rbq/tst/rbqlua/funcs/rbq_DrawBitmapText/tickTest.lua is working.
Created 'exp/dev/tst/testWriteGraphics.lua', which defines a tick script and a draw script for a script object. The draw script calls rbq_DrawBitmapText() for each tick, and the tick script does a graphics buffer dump and a redraw for each tick, for 10 ticks, but the text never shows up in the output files.
Looking at the output of the event scripts, it seems that the draw() event is called after the tick() event, for each tick. Also, it is necessary to call redraw() from within the tick event in order for the draw event to be called each tick. However, very strangely, if a second exportGraphicsWinBuffer() call is done in the tick event after the redraw(), the draw event is not called! That doesn't make sense!
Wait, calling the exportGraphicsWinBuffer() function does cause a 'renderPixmap()' function to be called. Is this clearing the 'redraw' flag?
Ah, think I see the problem: the paintGL() is being initiated by the renderPixmap() call, but all this is happening from within a tick() event call, and the script manager must be preventing the call to the draw script. If so, at the very least, a warning should have been reported. Yup, that's the problem.
Added warning to CRbqScriptObject::RunScriptFile().
This current restriction appears to prohibit testing of graphic contents through script object tick and draw events. One workaround may be to add a 'post message' hack to allow a tick event to request a graphics dump from outside of its executing code (similar to the recently-added 'scheduled delete' of script objects).
Was about to add command 'dev schedDumpGraphicsWin', and associated function in CRbqMainWindow() 'ScheduleDumpGraphicsWin()', when I realized that I could generalize it to a submission of a list of commands to the command processor instead. This allows script object tick event code to submit arbitrary commands for execution after the event code execution has completed, eliminating potential problems where one of those commands would result in the need to run some other script object event code (not sure, at this point, how much that would be needed, but the functionality will be in place). Currently, only the 'dev dumpGraphicsWin' command requires this functionality. In retrospect, the function to do scheduled deletion of script objects could have made use of this more generalized function.
Added command 'dev appendPteCmd' and associated functions. A fortunate bug in the cmd processor code also revealed the need to add a reentrant check for CRbqMainWindow::RunPteScheduledCommands().
Whew, after adding the 'dev appendPteCmd', the tick-driven graphics draw and write is finally working in exp/dev/tst/testWriteGraphics.lua!
rbq_DrawBitmapText unit test is finally passing! Wow, a new world record, 11 days (2004-05-25 to 2004-06-04), over the previous winner, rbq_RotateForceObjectNode, which took 6 days (2004-05-17 to 2004-05-22).
Created new unit test rbq_DrawOrthoText. To avoid code dup, will create a helper script object that will do the setting up of the graphics window to a specific fixed size. This will also simplify the test case code for the unit tests, as the tick function will just wait until the helper object has completed the setup.
Created helper so script 'rlutSetGraphicWin.lua'. Also, for the first time, in adding another function to the 'TestUtil.lua' module, I decided to make the function a member of a 'TableUtil' object, requiring qualification of the function call: TestUtil.getRlutDir()', because at some point, name clashes will start to be a real problem with the current 'global' function scheme. However, a transition to this better way will require huge code changes, although Lua appears to have many mechanisms for adding global names, local names, and putting module names in the global namespace (Lua book, Chapter 13).
However, this change also has consequences for the documentation scripts (alphabetical ordering), but that may not be too big of problem. We'll try it and see.
Changed rlut rbq_DrawOrthoText to use new graphic setup scheme. The test is working.
Added documentation of tick unit tests to Guide for Writing Unit Tests.
Created StringUtil.parseOptions().
Created 'tickTestDriver' script. Created 'runSubsetTests-postTtd.lua'. There is now a heavily duped-code problem between runAllTests.lua and runSubsetTests.lua. Probably will ditch 'runAllTests.lua' and create an 'all.testNames' file and call 'runSubsetTests.lua' with that file as a param.
'immediate' and 'tick' unit tests are both being run from 'runSubsetTest.lua'.
Added another test case to rbq_DrawOrthoText, to fill up the screen.
Created rlut 'rbq_DrawBlockArrow', 'rbq_DrawBox', 'rbq_DrawCylinder', 'rbq_DrawCylinder', 'rbq_DrawSphere'.
Ack. Graphic unit tests are failing now because the png format doesn't appear to be invariant. A file of a different size is being produced today compared to yesterday, pretty sure the actual image is the same. Not a major show-stopper, just more time spent doing research on png format and possible alternatives.
Spent gobs of time downloading, compiling, running sng (scriptable network graphics) to output sng versions of differing png files, and writing a little perl script to give the diff lines and offsets. Then, tried to spot the individual pixels in photoshop but couldn't. sng files appear to show a numerical difference of a single digit, indicating some kind of round-off error.
A brief search on the web didn't bring enlightenment. The next step should be to save as bitmaps to see if this error occurs again, though how often it will occur is not certain to me. If the error is resulting from some kind of processing in the png, the test could be changed to output as bmp, though it would be desireable to add some kind of compression with that because the files would be much larger than necessary otherwise. Another nice option would be if someone has written a graphical diff tool. If so, it might be possible to do a compare with that which would disregard diffs due to round-off error.
No luck on image-diff. However, I suppose I could write a dedicated program to do that on the images without too much trouble. Although it has already been too much trouble.
Downloaded png2bmp, compiled, it works. Now I can just call that from the test script and then do a binary compare on the bmp files. They currently differ by just 1 for 11 bytes in the file. Will write a small c utility to do a fast check: 'minDiffByteVal' or something like that, which will return a result code to the test script.
Updated the links page with some more molecular modeling mailing archives.
Submitted a project report to the Lua website about Ribosome Builder.
Did a little research on GUI libs for lua. wxLua looks interesting. One possible problem is that support for Lua 4.0 is disappearing. wxLua 1.4 still supports it, I will grab that one for the present time. Also, wxWidgets may be a good alternative to Qt to escape the current VC 6.0/Qt 2.3 bind.
Rotamer covariation!! Do it!!
Created rlut rbq_GetAtomTypeVdwRadius().
Working on more atom type-related rlut funcs. But there is a problem with nomenclature. I noticed that there currently is not a function to return the actual 'atom type' id for a particular atom of a dynamic residue. We need the function 'rbq_GetAtomType()', similar to the existing function 'rbq_GetAtomElement()'. But there are already several functions 'rbq_GetAtomTypeFromName()', 'rbq_GetAtomTypeName()' and 'rbq_GetAtomTypeVdwRadius()'. In all of these functions, the 'Atom' in the 'rbq_GetAtomTypeXXX' refers the 'AtomType' type, not the actual atom that possesses a particular 'atom type'. It's very confusing, but I'm not sure how to sort it out. The term 'AtomType' should probably be changed to something else, like 'ChemType' or something. But can't do that right now. Will have to think about it. For now, will just have to try to deal with it by putting sufficient comments in the api docs.
So this work will involve actually adding two more rbq-lua funcs, pushing the rlut progress back somewhat! Needs to be done, though. This also brings up an additional problem that I've noticed in the rbqlua api, that a number of current functions only info for static atoms, not dynamic atoms. Of course, the force object atom properties are provided indirectly through rbq_GetForceObjProperty() and the high-level lua lib functions getForceObj_XXX. Actually, probably only one for now, don't strictly have to add rbq_SetAtomType() at this time.
That can be done in a non-runtime way by adding more definitions to the atomType files, though that really must be documented in the whitepaper soon.
Created rlut rbq_GetAtomType(). It works.
Create rlut rbq_SetAtomTypeVdwRadius(). In so doing, it becomes apparent that there needs to be a way of restoring the compiled default values. Currently, there is no way of doing this. If this unit test fails during a run, it will continue to fail until the program is restarted.
Adding rbq_DrawOrthoQuad, which is needed for transparent billboard display in the schematic sims. This caused me to review why I had to implement a separate transparent drawing pass in the main render function. After reviewing the code in CWkSmoothColorMesh::DrawTransparentTris(), it's because I had to disable the depth mask by calling glDepthMask(GL_FALSE), prior to calling glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA ). I did this because I didn't know much about transparent rendering and didn't have a lot of time to learn about the best way to implement it. It seems that a whole separate rendering pass is the wrong way to do it, and it doesn't even work properly with multiple transparent surfaces.
A brief search showed some posts on gamedev.net where people are doing it this way (a separate, subsequent rendering pass for transparent objects). Also, the sorting of transparent triangles from far to near in drawing (painter's algorithm) is mentioned. I am doing this for individual pdb objects, but not globally, so that is why my transparent rendering is not always correct. And given the existence of transparent drawing for script objects, implementing a global draw ordering of triangles will be more difficult.
Read a nice paper on transparent rendering. It makes the clear point that if the depth buffer is turned on, then an opaque object that is farther from a transparent object, but rendered later, will not show up. It also mentions the 'draw solids first, then depth off, then draw trans objects'. So the current method does not seem totally wrong , except that somehow global sorting of triangles needs to be done for the transparent pass. The only way that I can think of at present is to change the actual drawing code to some kind of 'submit these triangles to a drawing list' operation, and then the top-level draw trans function does the sorting and actual drawing of triangles that were submitted by the object hierarchy. I suspect however that with the newer graphics cards and engines and dynamic lighting and such, that this whole line of talk may be completely archaic and irrelevant. Time will tell.
Added some notes on transparent rendering to the whitepaper.
rlut rbq_DrawOrthoQuad is working.
There is a bug in OutputCurSelTorsions.lua, when run on 1tra.pdb. Wow, the similarity of getAtomParName() and getAtomPartName() is totally confusing. Arg. Renaming: 'getAtomParName()' renamed to 'getAtomPdbName()'. Thought about renaming getAtomPartName() renamed to 'getAtomTrimPdbName(), but reconsidered because there needs to be a function that returns the partial 'atom-only' part of an atom name, in contrast to 'getAtomName()', which returns the full name.
Fixed bug in getAtomPartName(), dihedral/torsion scripts are working again.
TODO: create a script in the 'Geometry', 'Torsion' menu to create an html report for dihedral angles of current selection (just need a driver for the exp/dev/report/DihedralReport.pl).
Updated ScriptUtil.parseOptions to handle short options, long options and non-options.
Torsion Angle Frequency Report script 'lua/TorsionFreqReport.lua' is working. I think the next step after this was something I was thinking about a while back, a way to generate a frequency report from a list of objects. This would allow the accumulation of statistics for a larger pool of conformations.
Thinking about how to implement this 'cumulative' version of torsion frequency calculation, this shouldn't be too hard. The current version creates a 'torsion angle' file from the current selection, which has a line for each residue, containing the torsion angles for that residue. In the cumulative version, we can just define a 'torsion angle' file that we append residue records to, for all residues of all models in the source model list. This file should be able to be input to the existing perl script 'MakeTorsionFreqTable.pl', without any modifications. The only change would be that column 1 would have the full dotted name of a residue instead of its numeric index, but that shouldn't be a problem. The only remaining issue is what to name the new script. Something like 'multi-model torsion freq report', 'mmTorsionFreqReport.lua'. That works.
This work is preparatory to taking another shot at accumulating g-rotamer stats (general rotamers, 324 possible compared to the 42 proposed by (Murray et. al, 2003)). This involves all possible combinations of (P1, z, a, b, c, P2) where P1 and P2 are 3' and 5' pucker and (z, a, b, c) are the 4 torsion angles that can be in one of 3 states: m, t, p (gauche minus, trans, gauche plus). The only thing that bothers me is that the Murray table gives a constant value for epsilon, but the frequency stats I just generated for epsilon for 1fka show quite a bit of variation. It makes me curious what feedback there has been to the Murray paper. 0 hits on Usenet. You know, it's pretty damn sad the way the Usenet has completely disappeared with respect to scientific discussion.
Added a little about torsion angle anaylsis to the whitepaper.
Created script 'lua/mmTorsionFreqReport.lua'. Starting to get a better idea about where I want to go with the rotamer thing. An immediate project would be to try to produce torsion-directed conformational change of 2 linked residues from one rotamer to another, for every possible combination of the 42 m-rotamers (and possibly, some additional well-represented g-rotamers). This would be 42 * 41 = 1722 simulations if bi-directional changes are investigated, or only 861 simulations if only uni-directional transitions are done. The relative difficulty or ease with which a subset of the transitions may occur can then form the basis for further investigation, especially the coordinated transitions involving multiple successive residue suites.
mmTorsionFreqReport.lua script is working, with only minor changes to exp/dev/report/MakeTorsionFreqTable.pl'.
So, before proceeding with this project, I need to get solid g-rotamer and m-rotamer stats from a large source of nucleic acid structures. This will require a rotamer version of the mmTorsionFreqReport script that has just been created. There are existing rotamer perl scripts in exp/dev/report. I need to create another lua marshall script, 'mmGrotamerFreqReport.lua'.
Started writing a script to automatically convert bridge scripts for 1gix/1giy to other pdb models, but I think it will be too much work and will be faster to just manually create scripts for additional models such as 1ffk, 1fka (at least for now).
Creating bridge-select scripts for 1ffk. Note: L19 (involved in bridge b6) is radically different in size and location in 1ffk compared to 1giy. Instead, L24E is the 1ffk version of 1giy L19.
This changes requires adding arg support to the ScriptDefs.ini file. Hope that won't break anything. Actually, it doesn't. The current mechanism already supports additional arguments after the script file name.
New version of bridge selection is working. Also added functionality to select all bridges.
Looking for newer models of ribosomes, found 1pns, a model of E. coli 30S, stated as 8.70 A resolution, but the model used full-resolution atoms. Nice. Of course, Ribosome Builder should have a similar full-atom capability.
Created a dynamic portion of 1j5e.A helix 27 and looked at the initial forces. There appears to be a bug in covalent force calculation because the initial structure shows a lot of covalent force arrows, all aligned in the same direction. There should be no initial covalent force on a newly-created dynamic structure which has not been allowed to move. I wonder if it is just a round-off error? Started initial control sim of selected portion of helix 27 on ribo4. After 140 frames, looks fairly stable.
Redefined rbq_DrawWireArc() to take explicit ptCen, vLook and vUp. It appears to work.
Drawing of torsion dials is finally working.
Found the problem. Was calculating fraction of 360 degrees inversely in rbq_DrawWireArc(). Fixed. Now there is another bug where the arrows flip direction. Actually, this looks like a bug in the DihedralUtil.lua code, because when the flip happens, the reported torsion angle flips from 0 to 180. I thought that was an old bug which had been fixed! Arg! Looking at old log, on 2004-03-26, the description of that old bug seems to be exactly the same problem. The cause was a zero cross-product. In this new bug, the problem seems instead to be from vectorAngle().
Fixed. Hope this is the last of the dihedral bugs! Torsion dials appear to be working. They appear to be very successful in indicating what the torsion springs are supposed to be doing.
Working on ribo5 to simulate inter-subunit bridge interactions for Hennelly paper. Will simulate all bridge residues dynamically by creating inter-residue meshes using new script object 'interResSpring.lua'.
Along the way, needed to add support for dynamic residues to script 'lua/ReportSelResDistance.lua', but in so doing, found bug in getSelectedForceObject(). Arg. Has something to do with the fact that there are multiple dynamic segments with the same chain name. Failure is in CRbqDoc::GetDynamicSegmentIndexFromName(). Actually, it's in CRbqPdbMol::GetDynSegMhIndexFromName(), which doesn't even consider multiple dyn segs per chain!! TODO: add creation of multiple dyn segs per chain to dynamic model unit tests!!!
Note: the bug above doesn't show up during mouse selection, because that is by index instead of name.
Hopefully fixed bug by adding 'CRbqPdbMol::GetOrdinalDynSegName()' fixup in CRbqDoc::GetDynamicSegmentIndexFromName(). Bug appears fixed. ReportSelResDistance.lua is working for dynamic residues in multi-segment chains.
I've worked out a way to systematically create the inter-residue spring-bonds for the subunit bridges. Adapting from 'initTorsionSpringsFromFile.lua' and the associated .ta file, I will create the script 'initLoopMeshFromFile.lua' and an associated .loopMesh file, which is simply a list of records where each record specifies the start and end static residue full names for dynamic segments that will be the loops. The script will create all the dynamic segments and then define inter-res spring bonds that connect the endpoint residues of each dynamic segment with the N nearest neighbor endpoints. This should produce a fairly stable yet efficient mesh.
Finished 'initLoopMeshFromFile.lua', defined bridge and adjacent residues in subunits, started first inter-subunit mesh sim on ribo5.
Next work on modeling bridge interactions is to produce some kind of quantitative analysis of bridge residue proximities over time. A way that this could be done is to define an approach vector of one subunit relative to another, then incrementally move the subunit along this vector and calculate the steric energy between bridge residue pairs. Will create a script called 'bsa.lua' (for bridge steric approach).
The way that things are currently done is that, for backbone output, a function, 'ExportGraphics()', opens a new .pov file from param filename, outputs some common boilerplate povray code, which does required includes for povray functions such as math.inc, sets the background color, camera location and orientation and light source. Then, after declaring the start of a union block, the file handle, along with a param export graphics object, is passed to the document object, which traverses the pdb object hierarchy to output triangle meshes and cylinders to represent the nucleic acid backbone (proteins are not currently supported).
The same thing is done for surface mesh output, except that the pointer to the 'CExportGraphics' object is pointing to a CPovRayExportGraphics_PdbSurf object instead of a CPovRayExportGraphics_NucleicChain object when entering the CRbqCmdProcessor::ExportGraphics() function.
The document object just iterates each pdb model object, passing the param output stream and CExportGraphics ptr. Inside the CRbqPdbMol function, a dynamic cast is done to detect whether it is a CPovRayExportGraphics_PdbSurf object, in which case the pdb surf object member function 'OutputSurfGraphics()' is called. Otherwise, each chain in the pdb model is iterated, calling the 'OutputGraphics()' function of the param CExportGraphics object, which in this case is a CPovRayExportGraphics_NucleicChain object.
This current implementation is quite messy and needs to be redesigned.
Combined surface mesh output with nucleic chain backbone output. Also fixed some other things in the POVRAY code, such as the view problem that existed before, where the position of the graphics in the OpenGL window didn't quite match that in the POVRAY image. I found that an angle of 75 in the POVRAY camera seemed to produce a good match.
The POVRAY export is much more usable now. One more thing to add would be transparency to surface meshes.
Did unit test run, all passed. I think it may be time to do a new file release, perhaps after adding transparency to povray export.
Added transparency to surfaces in povray export, it works.
Added coloring of each residue to povray nuc chain graphics. However, the problem with that is that the current implementation defines a nurb ribbon segment from the backbone atom of the current residue to the backbone atom of the previous residue, and when a single residue is colored differently from the adjacent residues, it looks weird. The expected coloring should go from the midpoint between cur res and prev res to the midpoint of cur res and next res. So this would require calculation of nurb segments involving the next residue. More work.
Actually, because the phosphorus atom is the first atom in the residue bb chain, setting the color using the previous residue color should work (nurb seg is from current phosphorus to prev phosphorus).
This is still a problem with missing bases (backbone atoms only), because the backbone ribbon requires an orientation. I may implement a second version of backbone export using povray sphere_sweep, which would also work for proteins.
Added new version of backbone rendering in povray export using sphere_sweep object, it works! Added covalent output, it works. Added spacefill output, it works.
Generated another picture of the 702 region in 1j5e for Scott, it looks much better now.
Installed winzip8 on ribo5 to create a distribution there using the personal self-extractor. TODO: replace with a free sfx version.
Fixed GetPos() bugs.
Building and testing rel can 0.3.2 on ribo5. One big problem: the 'Error: already running script...' message (and companion 'Error: failed running script...' message in CRbqScriptObject) is slowing things down when running the translation init demo. Need to change warning so that it is only given once per a particular currently-running script.
Done. Trans init demo re-runs without problems now.
Downloading povwin3.6 to ribo5 to test rb povray export. It works on a test render of 1fka and 1j5e, but there is a strange little artifact in part of the image. I think I will release this version.
Also ran the unit tests, and all non-graphics tests appeared to pass, and a check of the final graphic image appeared ok. At some point, when there is time, I will add the .bmp image check because I think the diffs are just do to math roundoff errs.
Did file release of version 0.3.2.
Trying to get a better feel for how helices pack against other helices. It's hard to see things with full vdw radii, but otherwise, it's hard to tell exactly what is packing against what. It would be really nice to discover some simple motifs or patterns of stacking, involving individual atoms. One way to start with that might be to write a script that, for any given atom of a residue, will output the nearest three atoms of a neighbor packing residue. Then correlations could be done between pairs of atoms, and their 'tri-nabor' pack atom pairs, and so on. Also, could output pairs of atoms that are actually 'touching', though this depends on how the vdw radii are defined.
I wanted to look at more examples of helix packing, and this led to the problem of trying to align additional models of the 30S, which requires sequence maps and helix alignments. After adding a little discussion of sequence and structural alignments to the whitepaper, I am thinking that some scripts should be created to automate the current process, which is a manual and time-consuming one. The first step would be to create an unaligned sequence file of a pair of chain sequences. This would be done by running a script in the program that has the desired models loaded. Something like 'OutputPairChainSeqs.lua'. This would then serve as the input to a pair-wise sequence alingment program such as globals.
Created function 'getNucleicChainSequence()' in SeqUtil.lua. Starting on 'lua/CreateSeqMap.lua' script.
Fixed bug in povray export for invalid residue sequence number (-1 in 1giy, chain 1, residue 0).
Finished initial portion of 'CreateSeqMap.lua' script, but then, in using the 'globalS.exe' program that I had compiled on 2004-10-13, it is crashing. The third-party code seems a bit of a mess to me, and I don't want to take any time to clean it up. Added some exit() statements to bail out instead of crashing when arguments are missing. Also, I had forgotten that I had written a script, 'aln2fasta.pl', to convert the output of the globalS alignment into a fasta alignment. Fortunately, all of these 'manual' steps will be encoded into the 'CreateSeqMap.lua' script. Still, the entire operation appears rather brittle at this point because so many disparate command-line tools are used. In GNU/Linux, the error-handling and chaining could probably be handled just fine, but in DOS it will probably fail miserably in edge cases.
Well, there is already a problem using the brain-dead DOS command-line. The current implementation of globalS outputs to stdout, which I cannot redirect in the current implementation of runProcess(). Rather than trying to hack a redirect, I think I will just hack the globalS code to output to a param filename.
But now the 'run all tests' test driver is broken. Ah. Because I changed StringUtil.parseOptions(). No problem.
TODO: update graphic unit test code to check diffs, get rid of hard-coded .hlx file refs in _SeqUtil.lua.
Idea: ligand-receptor docking using the 'single contact sphere-rotate, two contact axis-rotate' strategy.
Currently, the following SeqUtil functions reference 'f_HelixData':
Changing these refs shouldn't be too much of a problem in this particular case, but it leads me to consider adding tests to verify the continued function of scripts, in addition to the existing unit tests for the rbq-lua api. These wouldn't be called unit tests. Instead, they might be organized by category, and called 'feature tests'? Such as 'sequence feature' tests for sequence-related scripts. Should these be called 'regression tests'? Or 'scenario tests'? Or 'category tests'?
Using the rbqlua test driver structure as a template, created new feature test subdir 'app/rbq/tst/seq'. Created test 'app/rbq/tst/seq/tests/CreateSeqMap/test.lua'.
Added rlut rbq_GetSelectionExtent.
Added rlut rbq_GetTotalSelectedChains.
Wrote a description of the Helix Alignment Algorithm for Scott H..
Did full test run for rbqlua tests, 88 tests run and passed.
While coding the rlut rbq_RotateSelection, I noticed that there is not any current specification of the requirement for vector normalization for any of the rotation functions in the class hierarchies and libraries. Tracing it all the way back to grCommon::GetRotatedMatrixOpenGLStyle() in the wkGrMatrix lib, I see that the vector is normalized inside the function. Adding an explicit specification that the vector need not be normalized would be a lot of work. And perhaps the current absence of this is the expected behaviour. If a param is required to be normalized, it should be stated explicitly in the specification. In addition, for possible optimization, there perhaps should be another version of the function that doesn't do the normalization, or, to avoid a zillion function dups, perhaps a state, such as that in OpenGL, which enables auto-normalization (I believe in OpenGL, it doesn't check for normalization by default).
Also, that would be a nice thing to add to all rluts which use vectors that possibly might need to be normalized: test cases which pass non-normalized versions of the parameters.
Finally finished rlut rbq_RotateSelection. Disturbingly, the inverting atom position error in rotating a dynamic segment by 180 degrees about a vector orthogonal to the atom position vector is greater than 0.001. It does pass with an error limit of 0.01 however. Presumably, this is an accumulation of numerous small errors in the vector, matrix and center-of-mass calculation code, but it should probably be checked at some point.
90 rluts running and passing.
Next is the issue of supporting only a selected portion of a dynamic segment for export. The current implementation only supports the export of the first fully-selected dynamic segment. It should be easy to add a flag specifying that dynamic atoms should be exported only if they are selected. Done. It works.
The reason that I added this capability was to enable export of single atoms to a PDB file, with the intention of using them for testing of the forcefield. However, I'm not sure if the current implementation of force objects permits creation from partially-defined residues. If so, it should be added, because single atom-pairs would be the best way to test the forcefield functions.
BUG: when a single node of a dynamic model is selected that doesn't rotate, or when multiple nodes are selected, they will not rotate, but no error message is displayed to the user.
I'm also modifying export of dynamic segments to support composite (multi-model) export. At some point, the static model export should be updated to support this as well, but that will be a little more complicated because the current implementation of static model export supports exporting chain descriptions. The dynamic model export has the flexibility of composite export, but this isn't compatible with maintaining existing chain descriptions for 'all-or-nothing' static model export.
I should add an 'export' section to the feature tests in the 'tsts' subdir.
After these changes, did rlut tests, 94 run/pass.
Looking at the requirement for creation of dynamic residues, in CRbqDynamicSegment::ValidateForDynamicResidue() it requires that atoms in the 'extra-bond-definitions' must be present. Why was this made a requirement for dyn res creation? Looked back in the old dev log, on 2003-11-13, the reason for this was that hydrogen atom addition would fail because some atoms were missing. If extra bonds were not added to an atom because its partner atom in an 'extra-bond-definition' record was not present, then the correct valence of the atom would not be determined, and hydrogens might be added incorrectly. So, currently, if this validation fails, the code just drops this residue completely. I'm thinking that it should be fairly straightforward to just drop hydrogen atom addition instead, for 'incomplete' residues.
However, would it be even better to do the enabling of hydrogen addition on a 'per' atom basis, so that we could have a singly-defined hydrogen bond for testing? This is more problematic, and since there is a current capabilitie to disable individual hydrogen bonds in the forcefield, that could be done instead. Note: currently, only the DEBUG version supports this, and it isn't available in rbq_SetForceObjProperty().
Adding code to just skip hydrogen addition instead of dyn res creation if missing atoms. However, in creating for a test residue with just two atoms, there is now a failure in creation of CRbqTorsionGraph This is due to non-bonded atoms in a residue. Trying one P atom per residue, and trying to create 2 dyn residues, now getting a failure in CRbqNodeBsjForceObject::SetPrevResBondData(), so we have to handle that as well (dynamic segments with adjacent residues that are not bonded to each other). Though, for testing, could have 2 separate dynamic segs with one atom each?
Successfully creating 2 dyn res in separate segments, each with a single P atom.
However, in devising this test, this brings up another interesting thing in that the total force on the single force object should be zero, and the deviation from this is a good indication of the numerical errors. Currently, for the test object 1tra.a$.G1, the total force after 1 timestep is reported as (-0.00000009 0.00000009 0.00000021), which is fairly close to zero (nearly within the expected precision of a 16 bit float (is it 16 bit currently?).
Tracing into CTorsionBodyIteFcon, the pair count is 172 instead of the predicted 174. That may be a distance threshold effect, retrying with larger threshold. No, that didn't do anything. That threshold value is a residue-level param anyhow.
Then, reading Lowe's Pipeline blog entry about biochemistry processes that are modelled with mathematical equations that have these high-exponential terms with tiny coefficients that only affect the result in unusual circumstances to reveal some buried biological effect, this led me to try to imagine a way to connect some particular environmental context of a macromolecular system to these low-weight exponential terms of an abstract mathematical equation.
Did the command 'ff fo report torsionTorque' and got a listing of 96 torques, as expected for 48 torsion interactions. Could just add the command 'ff fo report torsionBonds' to list the torsion bonds for the selected force objects. Done.
Looking at the report, I found some missing torsion bonds, but there are also some extra ones! These are bonds where the endpoint atoms A and D are the same, but atoms B and C are different. For example, in the 6-ring of Guanine, there is (C5 C6 N1 C2) and (C5 C4 N3 C2). The existing torsion code must be ignoring a torsion bond if one with the same endpoints already exists, but this is incorrect behaviour, because torque should be applied about both torsion bonds for those pair of atoms. This is a good bug find from developing an rlut.
For covalent interactions, there are 39 single covalent bonds visible. 44 are reported, so there must be 5 additional double bonds. Verified.
We finally have a working version of rlut rbq_GetTimestepInteractionCount, although only for a single forceobject. Enumerating the theoretical counts for multi-residue tests will take a bit more work.
Fixed ff cumulative interaction stats. Ran rluts, 100 run and pass.
Note: steps for ssh on GNU/Linux:
Starting doing some thinking again about the schematic sims and how to design the simulation portion and the annotation portion in the best way (model events vs. presentational events). Currently, there is an interpolated presentational event script object (spIpe), but this seems rather kludgy. I'm thinking that there needs to be a clean separation between the code that implements the model simulation and the code that implements the presentational stuff.
We also need an easier way to crank out a lot more schematic sims. I think that a good way to go about this is to create some tools that can help to generate the needed files. Such a tool would let the user define which models are to be loaded, then to define a set of interpolated sequences, then to define camera shots and billboards, and ultimately, to do much more sophisticated presentational events during the model simulation.
Added 'diffColor' functions, they work.
My plan of how the game should work has been evolving. Initially, I envisioned a rather complicated requirement of the user aligning each fragment until it is nearly in the complete correct position and orientation, but now I realize that it could be a lot easier, and still educational, to require the user to choose a fragment and then just put one of its endpoints in proximity to an endpoint of the correctly-assembled portion, and then do a 'submit' action. If it is the correct fragment and endpoint, the game will then do an animated alignment of the fragment, with accompaning sound at some point, for a nice 'reward', or play an annoying buzz for failure. The scoring can be simply based upon the number of correct successive submissions.
Next: starting on 'dock' script object.
Created the docker script object, which is currently docking and setting color of docked frags each timer tick. But I have encountered a problem, where the selection coloring mode is staying locked. It results from the selection that the script object is doing each timer tick, but I don't understand why. It's annoying, and potentially time-wasting, so for now, I will just do conditional dock and selection/coloring for fragments that are not in the docked position. That fixed that problem, but still worrisome.
Interpolation works! Just a simple cut and paste from the simpanel interp code. We almost have a game!
More evolution of how the game should work. Found that a black background works much better, and also setting the orig model as a transparent surface to show the envelope. Also, it seems too hard to expect correct sequential assembly, especially for newbies such as myself. So instead, I am allowing all submitted fragments for docking, in any order, and will just calculate the score based on proximity to destination. Finally, as far as creating helix annotations as clues, I think for now we can just draw annotations for helices that are fully contained within fragments. This will lower the current performance hit for lots of annotations, and also solve the problem of having to add 5' or 3' side designators to the annotations.
In developing the Rbg game, I started making extensive use of a module-level object (RbgGame) to hold data instead of in script objects. It caused me to realize that this could be a better alternative to the cumbersome script object variables for all script objects. Each script object could be created as a table object, held in an array in a module-level lua variable. Data access could be much easier and powerful. Possible downsides include somewhat less visibility and access from the user level compared to script object vars, although presumably inspection routines could compensate. Also, there might be some problems in freeing unused data after script objects are deleted.
Added new game function to rbg game. Now it is fully functional. Also, realized that I didn't add random rotation to frags, just random translation. This actually makes it easier. Need to hook random rotation to higher difficulty levels. Also need to add helix labels soon, as this will help the user to learn the structure. Also, add mrd color (most recent docked frag color, as distinct from remainder docked color).
Added semi-transparent backgrounds to text messages in rbg game. Added some helper functions to do some of the calculations, but the process is still kludgy, requiring separate 'matching' functions called from the 'drawTransparent' event for each text function called from the 'draw' event.
Added helix labels to fragments in rbg game.
Released version 0.3.3.
Looking at the directory structure, I'm thinking changes need to be made. First and foremost, the distribution and install mechanism should result in the creation of a subdirectory with the program and version name, e.g 'rbuilder-0.3.3'. Second, we should ditch the 'bas' directory on the development machines. The base directory is implicitly the 'rbuilder-XXX' subdirectory. Second, the 'app' and 'lib' subdirs, which are src directories, should be explicitly moved into a new 'src' subdirectory. Finally, the 'exp' directory should be maintained explicitly as a subdirectory of the base directory.
Adding keywords to my existing database of references.
Added 'command reference' html files to project web page, as User Manual and other pages referenced those, and were showing a broken link. However, need to come up with a more comprehensive way of updating these kinds of files on the project web page so that they are in sync with the latest build, as the current manual method is not good at keeping everything up to date.
Videogame boxing is a great caffeine burner-offer.
Added AnnUtil.setBillboard() and supporting billboard code.
So the time has come for me to use some of the object-oriented capabilities of Lua. I am comfortable enough with the language not to be bothered by this additional complexity. Read the old version of the Lua book (around when Lua ver. 4.0 was used). This version mentions two ways of creating objects: cloning a table and using settagmethod(). The table cloning method is simple, but uses more memory. The settagmethod() is fairly simply to use for the specific situation of adding delegation for table element indexing, but I believe that because it is a global effect, it could have a significant slowdown of the existing code because after setting a delegate function, every table index operation for all the current Lua code will go through this function. So I think I will stick with the clone method for schematic sim objects for now. Read a little bit of the newer version of the 'Programming in Lua' book, which is for post 4.0 versions, and it talks about doing things a little differently, using metatables, which I suspect allows for more specific, localized overriding of table index lookups, but I am not about to upgrade to a newer version of Lua at this time.
Ack. For the first time, I am running into lua file load conflicts, where ScriptUtil.lua needs to load AnnUtil.lua and vice versa. Why is this only happening now? Is it because I am doing a dofile() at top of _AnnUtil.lua to separately load AnnUtil.Sim.lua?
Ok, hardcoded the path for the dofile(), which is really ugly. Now having problems with function declaration 'AnnUtil.Sim.new()'. Can we not nest package names? Changed to 'AnnUtil_Sim.new()', did 'AnnUtil.Sim = AnnUtil_Sim;' at top of file. Is this the way to do it or is that a hack? Anyhow, now it compiles.
AnnUtil Billboards are working.
Since the console commands are used to implement the lua lib commands, then the bug should show up there as well. Why didn't the unit tests catch this? Answer: the currently defined unit tests only cover the rbq_lua interface functions, not lua_lib functions. Verified that setting the view up dir to (-1, 0, 0) will also change the view center.
Traced the code to CSimpleView::SetLookDir(), which does the actual setting of a new direction of the view. Rather than mess with this function, for now, we could just do a hack in CRbqCmdProcessor::Cmd_View() to adjust the view center to maintain it after changing view look dir / up dir. This should be a good solution because it is similar to what setView() in ViewUtil.lua does, where first the look dir is set, then the up dir is set, then the view center is set.
Did fix, it appears to work. Also, all non-bitmap rbq_lua unit tests are still passing.
Added 3 steps to Ban-2000 Journal Article Annotation, I think the new scheme will work. It will require periodic enhancement and creation of support routines as the annotation tasks are encountered, but as a result, the work of creating Journal Article Annotations should become faster and easier over time.
Added a parameter to the AnnUtil.Sim.incInterp() function so that each step in the sim can adjust the amount of time that is needed, relative to other steps. This is still a little ad hoc, use of an explicit real-time value like '2 seconds' would be better, but in general, most steps will involve a constant static delay of a few seconds so that the observer can read the billboard text and the associated graphical highlight events.
After adding a few more steps, we will be needing RTT functionality soon.
Adding a Camera to AnnUtil. Done. It's incredibly simple (2 functions), took only a few minutes to do, and it works beautifully. Such a contrast to the struggle in coding camera movements in the first schematic sim.
After a bit of thought, I realize that it should be simpler than that, all we need to do is a simple lua_dostring() or lua_call() from CRbqCmdProcessor::Cmd_Ann_Sim() and call a function in AnnUtil.Sim.
Adding code to load a sim at a param step.
'Goto Sim Step' is working!
Proper naming is hard. Trying 'sim:isFirstMatureInfoTick()' instead of 'sim:isFirstInfoDelayTick()'.
Adding display of interp state in graphics window, should have been simple, but there appears to be a bug in the Billboard display, the text and background quad are out of sync. Arg.
Ok, found the error, just wasn't noticeable until a bb with enough rows were being displayed, so that the row position error could accumulate enough to be noticeable.
Working on exp/dev/tst/testBillboard.lua to try to debug the problem, one part of the problem is that the command 'edit delAnnotations' is not truly re-initializing the AnnUtil.Billboard table of boards.
Added a fix for that where if the Billboard has to create the associated script object, it will also re-init the Board table.
Did one change of changing the hardcoded char height in CRbqLua.cpp : rbq_GetBitmapTextMetric() from 8x15 to 8x13. That helps.
After creating a working test of the Billboard to display all chars, it looks like the background quad just needs to be dropped down 3 pixels relative to the text. Not sure why, but we'll just hardcode that in AnnUtil.lua textAreaToPixels(). Ok, that fixed it.
One annoying effect: display of the interp info every tick messes up the playback rate. With a graphics update every tick, the sim doesn't need the 0.01 slowdown that was done there before, so I might just have the sim.tickStep() function do a redraw every tick. But this is still pretty ad hoc, what is really needed is some code to pace things to a fairly constant frame rate.
More tweaks to to sim controls. One small problem: the camera interp appears to have a slight inaccuracy, the destination point is slightly different from the specified one. This is actually a problem in the Ban-2000 runSim.lua script, and the Sim class, which terminate the step before calling the full fInterp = 1 value.
The obvious fix is to move the incInterp() to the beginning of every tick() function. Since we are no longer detecting first tick with sim.fInterp value (using sim.isFirstTick() instead, which uses sim.iTotalStepTicks), this shouldn't cause any problems.
Did it, sim works fine and camera interp is now ending at expected position.
The solution is probably to add another function to the AnnUtil.Camera class and have it do the actual positioning, because it will calculate each position independently, using an interp value. So we must remember to avoid the use of any 'relative movement' schemes that do not make explicit use of the interp value.
Renamed AnnUtil.Camera.setInterp() to AnnUtil.Camera.setLinearInterp(), added AnnUtil.Camera.setArcInterp().
camera arc interpolation works!
Now, we are needing multiple pauses for the events of a step. For example, in the current step 18 of Ban2000 sim, the first thing that happens is to throw up the new billboard describing a new domain, domain II of the 23S rRNA. Then, after a brief pause, the region is selected and appears to the viewer highlighted in yellow. Then, after another pause, the view begins moving, in an interpolated manner, to best show the user the domain. So there are, at minimum, two pauses required during this step, and possibly more will be needed in the future. So, there needs to be a better mechanism than the current 'sim:isFirstMatureInfoTick()'. One possibility is the more general 'isFirstTickOfPause( N )' where N is the Nth time segment defined for the step. Or, instead of pause, would a better name be 'isFirstTickOfSegment( N )'. In addition to the 'first tick' status, there also needs to be something like 'afterTimeSegment( N )', in order to properly call an interp function, only during that segment. A better name than segment is 'interval'. A set of functions should be defined, to do relational calculations of time intervals within a step, although we must be careful that all cumulative operations are performed, no matter how many ticks (and associated temporal locations in intervals) occur, as every step must work properly with just a single tick() call for interp value 1.
No, interval is no good, because it implies something bounded on both sides. Instead, we need something like 'isFirstTickAtOrAfterInterp()' and 'isAtOrAfterInterp()'. Of course, 'AtOrAfter' is much too verbose, as is 'GreaterThanOrEqualTo'. We could just use 'isFirstTickAfterInterp( N )', but this is dangerously imprecise, implying it should not be true for the value at N.
Other possibilities: get rid of 'FirstTickAtOrAfter' and use 'isFirstInterp( N )' for the single-shot. For the 'greater-than-or-equal-to', use 'isAttainInterp( N )' or 'isArriveInterp( N )', but neither of these is quite right. 'isPostInterp()' is better, but there still is some implication that it may not apply at the value N. 'isAtLeastInterp( N )' is pretty clear.
Then there is the problem of how to define the paramter. Is using an explicity numerical value verboten? Probably not, because the relative ordering would be more clear within the step tick() function definition. But then adjusting the value subsequently would be an ultra-hassle. So named constants must be used. Does 'isFirstInterp( sim.fInfoDelay )' work? Seems to, in conjunction with 'isFirstInterp( sim.fHighlightDelay )', but only if we assume that these two delays are always used in the same way, which they are not. Instead, how about numeric labels, 'isFirstInterp( sim.fDelay1 )', 'isFirstInterp( sim.fDelay2 )'? Still not perfect, but better than named delays and numeric values. These values could initially be assumed to be 0.1 increments, and could also be redefined subsequently if needed.
Ok, I think I'm going to go with 'isGainInterp( N )' for the greater-than-or-equal-to functionl. It's short and the meaning is fairly clear for a temporal context. 'Reach' is clearer, but not as mellifluous. 'isReachInterp( N )'. Hmm, maybe clarity should supercede mellifluousness. Yeah, gonna go with that.
Arg, 'isFirstInterp()' will be confused with 'isFirstTick()'! Now it must be 'isFirstReachInterp()'.
To do the actual 'first-time' check for a 'isFirstReachInterp( N )' call, will need to create a log of the fInterp value for each tick() call in the step. Then we would have to count of the number of log entries which have an interp value greater than or equal to the param interp value. This doesn't sound very efficient. That's assuming that the call can't have a side effect. If we allowed the call to have the side effect of setting a hash value, then determination would be easy, but then the call would only be valid once per step. This restriction seems reasonable, but might become a cause of bugs at some point. Ah, just record 'sim.fPrevTickInterp' value. Simple.
Added function Sim:isReachInterp() and Sim:isFirstReachInterp(), changed delay refs in Ban2000 and it works. Also added some pre-defined delays. However, one thing that is apparent is that only a single interp'd event can be done in any given step. Having two different interp events in a step, such as a linear view interp and an arc interp, separated by 'isReachInterp()' logic, would not work properly for a single full-value interp. So the delays will be used to separate multiple 'single-shot' events in a step, but not multiple 'interp' events.
Added AnnUtil.Camera:lookAtInterp().
Ok, the other problem with delayed interps is the 'jerk' that occurs when a camera interp starts at 0.2 and then goes to 1.0.
Added vUpDir option to AnnUtil.Camera.linearInterp().
Adding AnnUtil.LineLabels class. This should be very straightforward, basically just copy and slightly modify the AnnUtil.Billboard class to produce an annotation service for producing lines attached to text labels in the 3D graphics window.
Working on adding line lables to the 3 protrusions of domain II of 23S rRNA for model chain 1ffk.0. Found a bug in one of the sequence map scripts: SeqMap_23S_Ecoli_Hmari.lua reports that there is no mapped base for E.coli base 'C835', but there is one, plain as day, should be 'G928'. I note in passing that there is no unit test for any seq map scripts on bas/app/rbq/tst/seq.
This must be a bug in SeqUtil.lua translateMappedBases(). Found the bug, it is due to an implicit string to number conversion for one of the variables. Will fix by doing explict number conversion for the sequence position vars.
Fixed seq bug. Ran seq test, all passed (but need to define more).
Almost done defining protrusion helix callout of domain II step, but ran into a weird bug where using a fractional arg to incInterp() caused the camera arc interp to wack out towards the end. Why?
Looking at arc interp code, I'm really hoping that it's because I am getting the current view up dir instead of the initial one. Substituting initial up dir.
Yeah! That fixed it.
Created a new jaa: exp/articles/Carter-2000. This involves loading a new model, and at some point may require alignment with the reference models in std orientation, requiring the usual sequence of operations to create aligned sequence maps and helix files. It would be nice to have an automated process to do this.
The script 'CreateSeqMap.lua', created 2004-07, does the work of creating the sequence map. I need to add documentation of that to the User Manual. And then, only one other script, GenHelixFileForChain.lua, needs be run. One minor bug needs to be fixed: the current implementation for selecting helices is looking at hard-coded map files listed in SeqUtil. This should be changed to just look for an existing file in the seq subdir. Added helix file auto-naming to SeqUtil.lua function findHelixFileForChain().
Added some brief explanation of smap creation to User Manual. Testing creation of smap file for 1pns, I note that there is an ASSERT in CFastPdbObj in loading the model. This ASSERT results from an assumption in CFastPdbObj::CreateInterResCovalentBonds() that residue numbering corresponds to the residue order in the pdb file.
I want to keep the existing organization form, but I think that I can also achieve working version of the 'final' form by writing a script to process the current html files, in a sequential order as defined by 'next' links at the top of each document, and concatenate the contents into a single composite html file. This can then be read either in a web browser, or in the more traditional word processor application.
Created script 'makeThesis.pl'. First version was simple to do. It works.
Finally took the trouble to figure out how to do manual formatting of line lengths in vim: use 'gq', after setting 'textwidth' to desired line length. That ought to save me about 10 million keystrokes.
Actually, further examination leads me to think that the 'Fwd' and 'Bak' refer to the line connecting the filled vertices, not the triangle edges.
The hole is at the facial boundary between two cubes. The hole has a direction in that one cube is on the outside of the hole, and the other cube defines the 'inside' boundaries of the hole. The cube on the 'outside' is the 'Begin' part of the hole and the cube on the 'inside' is the 'End' part of the hole.
Created image 'HoleFill-130-203.bmp', which shows cases 130 and 203 adjacent to each other. Case 130 is an example of a PxFwd_HoleBegin and Case 203 is an example of a PxFwd_HoleEnd. It's starting to make sense.
After updating the \proj\graphics\chem\msurf\cv.lua case viewer script to examine the hole fill work, getting an ASSERT in wkGraphic lib for CWkGraphicObject.h : 101. This is in the SetName() function. Must not have been overridden in the class that implemented drawing the bitmap text for command 'ann insBmText'. This is a CWkBitmapText object. That's correct, no override for that function in that class, but doesn't really matter at this point.
Code to add figures works well, after I realized that doing percent scaling in html img tags wasn't going to work. It's better to have the pictures in their final intended size.
Finally finished marching cube stuff in dissertation.
Downloaded the latest version of FreeWrl on my linux box (version 1.08 pre 3) and tried to compile it, without success. Didn't want to put much time into it, but it looks like the same problem that I had trying to build previous versions - a bunch of dependencies for other software components that I have already spent too much time trying to co-install and build. It may also be due partially to my 'older' version of linux on that machine (Mandrake 8.1), but that shouldn't really be a problem. Looks like FreeWrl still needs some work in the user support department. That is one area why Ribosome Builder can be considered a success, at least for the platform that is supported. There is a pre-built binary that downloads and runs, with pretty much everything included (although the downloaded file is a little big, as a consequence).
Downloaded the latest version of the Cortona VRML client (version 4.2) and tried to install it for the Firefox 1.0PRE browser on my WinME machine. Didn't work at first, and then, for some strange reason, after I went to a web site with a VRML plugin detector (http://cic.nist.gov/vrml/vbdetect.html), it seemed to 'activate' the plugin, and I could then successfully load and run .wrl files in Firefox, which is cool. It used the OpenGL rendered (DirectX didn't work) and the performance was not bad, although nothing to write home about.
Did all this research on VRML only to realize that the vrml stuff in the rb project has not been used for a long time, only POVRAY export is currently done. What a dunderhead.
Starting on discussion of implementation of main application (and internal libs).
Creating a picture for a camera frame, found a bug in rbq_DrawBlockArrow(): the polys for an arrow pointing in -x dir have incorrect normals. Looking at the code, it looks like it fails for arrows pointing in the -x direction because in wkGLutils.cpp, glxRotateCurrentMatrix() is trying to do a cross-product on vectors 180 degrees apart, which will produce a zero vector. Rather than fixing glxRotateCurrentMatrix() at this time, hacked a fix in CRbqArrow to do a manual rotation. It works. But this fix should be applied to glxRotateCurrentMatrix(), because this function is also currently used in CGrAtmUtil and CRbqForceDisplay.
I think it just kind of 'evolved' that way, starting with the incorporation of torsion graphs into the CRbqPdbMol object, and doing some static model steric analysis with them, and then the force object and dynamic segment stuff being tacked on later. But it seems that it might have been better to design dynamic models as a completely separate collection within the document from the static pdb models, although there would have been numerous redesign issues resulting from that.
Right off the bat, in trying to develop this new interpolation, I am running into a bug in saving and restoring an rbd document. Saved exp/demo/mrnaWrap/model.rbd to a new directory, then on reloading, got an assert in CRbqPdbMol.cpp:3285 and garbage in the location of parts of the dynamic model. Don't have time for this!
Fortunately, this bug appears to be not repro. A subsequent save and load worked correctly.
In comparing the rbd files for before and after running the movie, the difference is that the after version is only saving 206 out of the 863 dynamic node transforms. Well, looking at the script that loads the demo, it is turning 'single-atom-nodes' off, because it says that the model was created that way. However, selecting and displaying indices for the loaded model indicates that the model has single-atom-nodes. At this point, don't know how much I will worry about it. Need to move on and this appears to be a bug related to that particular flag, which should not be an issue for newer models.
Exported wrapped model to exp/dev/mrna/interp/wrapped.pdb. Now there appears to be a bug in selecting the backbone of dynamic models. But this bug is not repro.
Created lua object DihedralUtil.Interp, to do torsion angle interpolation of dynamic residues.
For development, looks like we need to 'single tick' to AnnUtil.Sim in addition to single step. Done, and it helps.
Running into problems, it seems like dihedral angles are being returned instead of torsion angles!!???. Duh, yes, that was the problem (bug in newly-added getFoTorsionAngles() function).
DihedralUtil.Interp interpolation of all torsion angles of G23 is working. The cumulative effect of all torsion angle changes on the downstream residues is quite dramatic. Of course, this makes me realize that I am interpolating in the wrong direction. G23 needs to be interpolated to U20, not U26.
Interpolating to angles of U20, there appears to be a direction bug, originating from DihedralUtil.getAngleDiff(), which I just copied from torsionInterp.tick.lua. I don't know how well that script was tested. Another good candiate for unit test development. Currently, the only 'special category' testing that is done is tst/seq. 'tst/dihedral' or 'tst/torsion' would be an excellent choice for the next category. I think that the problem is that getAngleDiff() expects angles in the dihedral range [0, 360] instead of the torsion range [-180, 180]. But, would that really make any difference? Ach, dumbkopff! I was switching the params in the function call. I don't think it does make any difference. Ok, it's working now.
Now we have to figure out the position transform. Thinking about how to do a '3 atom alignment', I realize that we can do a dynamic residue alignment that is similar to the existing static residue alignment script (AlignSelectedResidues.lua), except that instead of aligning using two subsequent backbone vectors, we use two vectors from the first 3 backbone atoms of the dynamic residue.
Created script 'AlignSelectedForceObjects.lua'. It works.
Now the next challenge is to extract the rotation that occurs during this alignment, and convert it to a quaternion, which should allow for a positional interpolation. Reviewed code in exp/dev/simpanel/interp.tick.lua to see how the positional interpolation is done. It works by receiving starting and ending homogeneous transform matrices, along with associated center points, then translating them to the origin, then retrieving the quaterinions for them, then producing a slerped quaternion matrix, then translating that slerped quaterinion matrix to the linear interpolated position between the two param matrix centers.
The problem with simply applying this approach to the mRNA force objects is that the force object consists of multiple transforms, one for every atom. We need some kind of generic quaterion rotation that can be applied globally to the entire force object.
I suppose that we could obtain the slerped quaternion matrix and lerped translation matrix from a reference node in the force object, and then apply these two transforms to all node frames in the force object.
No, something else appears to be going on. It looks like something else is modifying the actual contents of the string somewhere. Ok, this is interesting. What is happening is that there was a script object initialization script, exp/dev/tst/testQuat.lua, which defined some vars, using the 'local' qualifier, but at the main level, so that I believe these vars were actually put into the lua global namespace. This script creates a script object with a tick function script, which is run every timer tick. In this tick function script, it calls 'setScriptObjectVar()' at the end of the tick function, to update a script object variable called 'fInterp'. This stores the value in the member data list of the script object, held by CRbqScriptObject. But then, after running the tick script, from CRbqScriptObject::RunScriptFile(), the function CRbqScriptObject::GetScriptGlobalData() is called, to implicitly update modified script object data. And this is where the fInterp variable is set back to the initial value. In looking at all this, it makes me wonder why we have both the implicit update of script object variables done by CRbqScriptObject::GetScriptGlobalData() AS WELL AS the explict 'setScriptObjectVar()'???
Also, there must be some global 'fInterp' value in the lua global namespace, though I'm not sure where at this point. It seems like the explicit setScriptObjectVar() statements are superfluous in the current scripts. Except in the precise case where we have the 'global' override problem seen here. So what could be done is to remove the 'GetScriptGlobalData()' call in the code. Hopefully, this will not cause any scripts to break. I believe that I have been doing the explicit update of all vars with 'setScriptObjectVar()' for some time now. But not 100% sure it is done for older scripts.
Ok, commented-out the post-script-run GetScriptGlobalData() call, I don't think this is going to break anything significant, but I will run the demos and unit tests as a check.
Quaternion-based interpolated rotation of force object G23 to align with U20 is working. Now, just need to incorporate the translation interpolation, and a scheme for doing full interpolation of mrna translocation is starting to take shape.
Things are becoming clear. What we need is a single function, probably in the GeomUtil module, something like getInterpTransform(), which receives a center point, a translation vector, a rotation, and an interpolation scalar. The function will return a homogeneous transform matrix that will represent an interpolated orientation and position of the center point. This single composite transform can then be applied to a set of force objects. It works!
Trying to get position and torsion interpolation of mrna working together in eln001. Currently, when they are both done in a single step, the moving residue reaches correct torsion alignment and correct translational alignment, but the final rotation is not quite right. When the positional interpolation is done in a subsequent step, with full torsion alignment pre-set, the rotational alignment is correct. What's going on?? Ah, I see that I was applying the torsion interpolation after the positional interpolation. Moving it to before.
That was it! We know have a working combined torsion and positional interpolation for a single residue of the mRNA.
S