+- phyphox Forums (https://phyphox.org/forums)
+-- Forum: App-Feedback (https://phyphox.org/forums/forumdisplay.php?fid=6)
+--- Forum: Experiment editor (new Blockly-based version) (https://phyphox.org/forums/forumdisplay.php?fid=15)
+--- Thread: My first experiences with the new editor for a larger project (/showthread.php?tid=2069)
My first experiences with the new editor for a larger project - Trikes - 12-30-2024
Hello Phyphox team, in the last couple of days is started to migrate an older experiement (see attachment of this post) from the the old to the new editor.
I believe, I understand the obvious benefits of the new editor. The Analysis tab has become leaner (less crowded) by moving container assignments to individual other tabs. Also the permanent display of the containers is increasing programming efficiency a lot.
Suggested preparation for larger experiments in the old editor before switching to the new editor, because the old editor offers a 2d layout while the new editor requires 1d editing: I believe the new editor respects the horizontal position from the old editor, but it seems that the vertical position is not recognized. Therefore the recommendation would be to "stretch" old projects in the old editor ahead of their migration to the new editor (basically this is resolving the 2d layout into a more linear arrangement of the blocks in the old editor). This way related blocks would be grouped together in the new editor. For my project, which I did not stretch this way ahead of migration, I found that related blocks of my 2d project have been spreaded over a wide range in the new 1d editor. It took quite a while to sort these things out in the new editor to provide in the UI some coding coherence to the user for more easily understanding the coding. I believe, smaller projects and projects with the typical top-left to bottom-right diagonal layout should not suffer much this way when being imported to the new editor.
Grouping blocks in the old editor provided some kind of implicit documentation (proximity of the blocks = "they belong somehow together"). Right from the beginning of working with the new editor I wanted to compensate the missing freedom of the 2d layout of the blocks by adding comments. I really would apreciate some kind of comment block (could be a Logic block) which would do nothing else but displaying some comment in the new editor. No, I don't ask for line numbers, but comments really could help a lot to structure and explain the coding. Yep, I noticed the possibility to add comments to blocks which pop up by clicking on the question mark icon then, but it is an extra click and not directly inside the "coding".
The migration between the editors works pretty well, especailly when taking into account that there is an implicit and enforced upgrade of the file format. From other threads in the forum I understand that attributes are omitted in the XML file created by the new editor in case their value is identical to their default value. But this way a project depends on the fact that the defaults never would change, I would prefer to also keep the default values explicitly in the XML to document them and avoiding the need to look them up each time. In my project the deprecated Clear attribute is essential. I hope that the automatic conversion to the replacing Keep attribute has worked correctly, at least I have not found obvious problems in this regard, but I still need to check things in detail and get used to the Keep thinking.
I noticed that the Empty input block has not been added automatically in all cases to the operation/data/logic blocks with undefined inputs? I like that Empty block because it adds some consistency. I have added it manually in the new editor to not leave originally undefined inputs "uncommented". I admit that I did not care about this in the old editor, which also might be the reason why the Empty input block was missing in the new editor at some of my blocks. I hope that I did not change the logic of the experiment by adding the Empty inputs later on manually, still need to check this. But I found that for some blocks I violated their syntax when adding the Empty input block, which I only noticed in the mobile app after scanning the QR code: 'Line 516: Value-type not allowed for input "length".' has happend for the Const and the Threshold blocks, for example. Hmm, XML line numbers ... maybe I would like them as some kind of mouse-over in the new editor to be able to relate the error message from the app to the actual source code. Or maybe having the XML import check of the app already inside the new editor? This would save some roundtrips between the PC-based browser (design time) and the mobile device (runtime).
When switching tabs in the new editor always the vertical center of the new target tab is being displayed. For smaller projects this is not a big thing, but for larger projects (my XML has 1400+ lines) this always means to manually scroll back to the blocks you are currently working on. A lot of enforced scrolling, I can tell. Maybe the browser could remember the position inside the tabs and jump to the last known position when visiting a tab again? This would increase usability a lot.
While grabbing some UI element with the mouse I got another one nearby or even "under" the one I intended to grab. By missing the scroll bar I dragged a block or started creating a new container. There is some general tendency, I feel, in UI elements to become smaller with regard to monitor resolution. It might seem looking cooler, but actually it is getting in the way of efficiently use the UI. In the end it becomes necessary to apply some accessibility functions to not get disturbed by such side effects. Oops, accidentally I touched the Back button on my mouse and lost my experiment in the new editor. The old editor had some emergency popup to ask if leaving the editor is really the thing I want to do. I was lucky to have saved my last changes in the new editor to XML and have not actually lost anything by accidentally leaving the new editor. Speaking of such UI related observations, is it possible to combine blocks in contradiction to their syntax? While not precisely dragging blocks on my large monitor I docked some Subrange blocks inside each others which was not making any syntax sense to me (I have not tried to load such an experiment to the mobile app, maybe such sub-blocks not covered by the original block syntax just would be ignored).
I needed to accept that my browser (Firefox) was not jumping between the occurrences of search terms. It was only possible to highlight them all and scroll through the tabs manually to visit the occurrences (introducing the chance to miss one or the other occurrence). Would be nice to have some search function inside the new editor for container names, which also would help in preparing container renames and deletions.
There was one hard but hidden error when importing my old project to the new editor. I found the symptom of this error only at runtime and the root cause after some debugging in the new editor. The operator of the If block was not recognized at all by the new editor and instead of getting the operator I defined in the old editor I got in all cases the = operator in the new editor. For fixing the If blocks I needed to manually compare all If blocks between the two editors. Better to not switch off the old editor completely, please keep it.
Container rename: The rename command is a little hidding in the container selection popup (scroll down to the bottom, next to the delete command, easy to select the wrong command). After a rename the UI does not update unless the blocks are edited somehow. To avoid manual cleanup in the UI the workaround is to save the experiment after a rename and load that saved file again, then all the dependent blocks in the UI do read the new container name. There also was once a strange error when renaming a certain container which introduced two times the same container definition in the XML file which caused an automatic renaming of one of the identical containers at the next XML load (I needed to edit the XML file to get rid of the duplicate definition).
Container deletion: After removing all usages of a container in the tabs and its final deletion (context menu) from the container list the container remains in the list of the available containers (container selection popup). The deleted container vanishes when saving the XML files and loading the file again. On the other side, if I explicitly delete a container from within the container selection popup it actually vanishes from that popup list (there is not yet confirmation popup when deleting a container?), but its block remains in the permanent list of all containers in the original UI of the editor; I need to delete the block manually from there (just an observation: when opening the container selection popup from this abandoned container block, no container is preselected (checkmark), because that container already is deleted and no longer available). Ah, when deletion a container which is still used somewhere the new editor won't save. Probably there is some check of the experiment against the Phyphox XML schema, but I did not noticed any error message in the UI, took a while to find that last remaining container usage on the View tab. After deleting this last usage I could save again.
Small wish for inside the container selection popup: My project has about 190 containers which is a lot scrolling in that popup. I wonder if there could be some jump in the popup to containers starting with a character by just pressing the character on the keyboard.
Copying the name of a container is possible? Have not seen a simple copy'n'paste option (for deriving dependent container names, for example). To copy the container name I need to open the container selection popup and to start renaming the container. At the rename popup I can copy the container name and cancel the operation. A workaround. Ah, speaking of the renaming popup: the old container name also could appear in the input field as a kind of input help.
Unattached blocks ignored at runtime: blocks at the Analysis tab that happen to be not directly bound to the Analysis block do not through any error at the XML import to the mobile app. Just an observation. I assume such abandoned blocks are just being ignored at runtime.
I understand that the new editor offers an easier access to create (small) experiments. For larger experiments I still love the "data flow connection lines" of the Analysis tab of the old editor which give a nice idea of how the experiment might behave at runtime, in my opinion, this data flow aspect is a little hidden in the new editor (currently I believe I would not have had the idea explained in this posting in the new editor, the "lines" in the old editor have been essential for me). But the new editor supports in a better way debugging an experiments (I could more easily identify semantic container duplicates and improve the experiment). I see why you switch to the new editor and for most users this is the right approach.
New editor rooky errors on my side easily possible, cheers, Trikes.
RE: My first experiences with the new editor for a larger project - Trikes - 01-02-2025
Update after some more work with the new editor:
Accidentally and unnoticed I created once more the subrange-subrange block hierarchy, because the second subrange block snapped inside the preceding subrange and not parallel to it. When scanning the QR code this was recognized by the mobile app and I got error message "Line 784: Unknown tag subrange". Snap point proximity might be an issue difficult to solve. But maybe there will be at one point in time the same check in the online editor as at import the experiment to the mobile app.
I saw that abandoned blocks (blocks not snapped in to the dedicated "tab" blocks like "analysis") will not make it into the XML file, isn't it? I was lucky because I kept the browser opened and was not really losing the abandoned blocks; this only would have happened if I would have reloaded the XML file. Well, this is a one time observation which better should be validated by somebody else. Maybe the editor could warn at XML export that there are unsnapped blocks which will be lost at the next XML export/load. There is no guarentee that the browser would survive the next OS update and restart.
With regard to the block comments "(?)"on the Analysis tab: (1) Similar comments are not possible at the other tabs? Would love to use them on the other tabs as well. (2) I started using these comments on the Analysis tab, but after having created 6 comments I found that they don't make it into the XML file. They seem being just lost, better someone is validating this observation as well. Maybe theses comments do not have a related attribute in the XML file, which might be the reason why they are being not offered at the other tabs as well.
Because comments are difficult the only way to introduce some kind of "documentation" seems to be the container names, but they are restricted in the UI to 48 characters? Same for formulas. Well, the blocks hardly fill the horizontal lenght of the screen, increasing that limit to 64+ would be quite nice.
The undo is cool, but the UI does not jump to the block(s) changed far away by the undo = difficult to see if it has worked or not in case the undo was at some part of the Analysis tab that is not being displayed when clicking on undo. Cannot tell how many undos are possible, I found at least 20 are possible.
My experiment as saved by the new editor seems to run from what I can see in the mobile app. The migration was much less effort than expected (to be honest, I had the fear I needed to start from scratch, but this was not the case). Next step is to analyse the exported data and the experiment snapshot to understand if the experiment still behaves the same as before the migration.
RE: My first experiences with the new editor for a larger project - Trikes - 01-10-2025
Update after some first executions of my migrated experiment:
A first execution of my experiment as saved by the new editor showed constant NaN for one column in the exported CSV file. I easily was able to find the root cause of this error in the container tab in the new editor. While setting the focus to the browser tab to which the new editor was loaded I had apparently done some wrong mouse click(s) and was not only setting the focus to that tab but also the static tag for one container. After removing that unwanted tag the export data of the experiment looked good again (by values in OpenOffice and visually in QGIS). In the new editor displayed tags can be edited right away, in the old editor changing these tags required some additional navigation step into the container popup. On my screen I have in normal display mode 78 container tag checkboxes displayed at once in which I could accidentally set a checkmark. But on the other side this direct display of so many tags supported me nicely in finding the root cause for the NaN. In the old editor the search for that root cause would have needed significant more time, and probably I would have needed to directly analyze the XML file instead. I assume that with the new editor it is less often necessary to go into the XML file and do edits directly in that file. The "instant edit surface" of the new editor is much bigger, you can accidentally edit something, but at the same time the directly visible "debug surface" is bigger the same way. Support for debugging is of higher value to me than the danger to do a wrong click. The linear block display style of the new editor matches more with the XML format in the Phyphox file than the 2d display of the old editor.
The linear (vertical) sequence of the blocks gives a better idea on the sequence of the commands within an experiment execution cycle. In the old editor I needed to edit the modules horizontal position (pixels) in the XML file to exactly control that execution order (horizontal position from left to right). Taking away the 2d design time freedom helped in better understanding / controlling the runtime behavior. My experiment still seems to be able to use a container ahead of its (value) definition, which is a key feature for me to split and build up buffers as I need them for calculations and for export. If the container has no value defined (in my experiment) in cycle 1 there is no negative impact of this missing value and in cycle 2 the calculations are apparently working fine by accessing the container value computed towards the end of cycle 1. This is special, I believe, with Phyphox, you have got the linear execution sequence of the commands establishing your algorithm plus typically about ~200 (assumption) more of the same cycles before the next GPS data set arrives at the typical frequency of 1 Hz.
What I find a little irritating in fluently reading the coding in the new editor are the two different block styles I noticed. For some math blocks the output container is being displayed at the top left area of the block, whereas in other blocks the output container is being shown at the bottom to the right. While reading through the coding the eyes do some slalom. Don't drink and code, don't get dizzy. Maybe the output channels could get another color scheme than the input channels. This could help nicely in concentrating on the data flow at the Analysis tab in the new editor, which could compensate a little for the no longer available "data flow connection lines" from the old editor. But don't become too fancy with colors, color blind people should still be able to notice some differences (e.g. in brightness). Individual containers should not get their own colors, which my approx. 190 containers I would not be able to notice the differences. Highlighting the output channels with some light orange background color would be perfect for me personally in guiding the eyes through the data flow, just to give an idea.
In the old editor it was possible to select multiple modules at once and duplicate them, have not seen this in the new editor, where you apparently need to duplicate individually one block after the other. Actions on multiple blocks at the same time happen not too often and would contradict somehow the snap-in feature of the new editor. Moving around modules in the old editor was easier, in the new one it is necessary to first break the connection between blocks before rearranging them and connect them again. I believe abandoned modules in the old editor made it into the XML file, while abandoned blocks in the new one won't. Due to the 2d layout in was possible in the old editor to see more coding at one screen, but for large projects and a 30% zoom the actual benefit was limited.
I have not measured it but I estimate that I invested something between 120 and 170 old editor hours in my experiment. The migration to the new editor, reviewing and optimizing the experiment took me about additional 20 new editor hours. The old XML file has about 1500 lines, the optimized new XML file has now about 1400 lines (and approx. Analysis 1030 blocks). The performance and code complexity optimization would have been much more difficult in the old editor than they were with the new one. I'm quite happy that the development invest one year ago was not lost (context to explain the invest: the experiment is recording GPS data and derives certain values from this data, one experiment execution saves me about 6 to 8 hours manual work in OpenOffice). What I avoided like hell was to click on the big trash bins in the new editor's UI. For me, deleting everything would be not the main feature of an editor. If I ever should accidentally click on these icons I hope that the cool undo would save me. ... Ok, I was too curious and I clicked and I was surprised. It does not delete everything? But what is it? The log of the undo buffer?
Next step for me is now to set up a real physical experiment in which I have certain input parameters like distances and velocity under direct control. This way I can calculate myself the expected output of the experiment and I should be able to see if my Phyphox experiment would compute the exact same values as the physical theory. But this is beyond the new editor, which helped in a great way to get there. Thanks!!! Goodbye old editor's module connection lines, hello new editor's development efficiency.