MedEye3d.jl
MedEye3d
Main goal of the package is conviniently visualize 3d medical imaging to make segmentation simpler
Some oficial introduction - you can skip it
Image segmentation in the medical domain has mul-tiple use cases. Most importantly it enables delin-eation of physiological and pathological structures,in order to confirm or reject some diagnostic hy-pothesis. In case of all segmentation problems, avery important step in evaluation of the segmen-tation algorithm output is visual inspection. Suchinspection enables researchers that are responsiblefor creating and or evaluating developed algorithmsto easily spot problems, and compare different algo-rithms in more detailed ways than can be achievedby usage of segmentation metrics alone. Howeverin order for such in development visual evaluationto be useful it needs to meet some usage criteria.It needs to be easily integrable to the program-ming language and libraries used by researchers.Performance of the tool must be adequate inorder to suit the iterative process of algorithm de-velopment and refinement.Representation accuracy must be sufficient forthe task at hand. It should not require an exces-sive amount of computational resources, in order tominimize its influence on running algorithms.Support for in memory data structures (arrays)should be convenient. Needs to provide possibility of simple manualannotations, on given mask and ability to controlvisibility and visual representation of given mask.Should provide also the possibility to display somemetadata in text format like segmentation metricsfor example DICE score.Ideally it should be also open source and welldocumented in order to enable users to modify itaccording to the task at hand.In order to address all of those issues in themedical domain and Julia language ecosystem thedescribed below package was developed.
Image below just represents limitless possibilities of color ranges, and that thanks to OpenGl even theorethically complex data to display will render nearly instantenously.
Below the functionality of the package will be described on the basis of some examples In case of any questions, feature requests, or propositions of cooperation post them here on Github or contact me via LinkedIn linkedin.com/in/jakub-mitura-7b2013151
Defining Helper Functions and imports
#I use Simple ITK as most robust
using NuclearMedEye, Conda,PyCall,Pkg
Conda.pip_interop(true)
Conda.pip("install", "SimpleITK")
Conda.pip("install", "h5py")
sitk = pyimport("SimpleITK")
np= pyimport("numpy")
import NuclearMedEye
import NuclearMedEye.ForDisplayStructs
import NuclearMedEye.ForDisplayStructs.TextureSpec
using ColorTypes
import NuclearMedEye.SegmentationDisplay
import NuclearMedEye.DataStructs.ThreeDimRawDat
import NuclearMedEye.DataStructs.DataToScrollDims
import NuclearMedEye.DataStructs.FullScrollableDat
import NuclearMedEye.ForDisplayStructs.KeyboardStruct
import NuclearMedEye.ForDisplayStructs.MouseStruct
import NuclearMedEye.ForDisplayStructs.ActorWithOpenGlObjects
import NuclearMedEye.OpenGLDisplayUtils
Helper functions used to upload data - those will be enclosed (with many more) in a package that Is currently in development - 3dMedPipe
"""
given directory (dirString) to file/files it will return the simple ITK image for futher processing
isMHD when true - data in form of folder with dicom files
isMHD when true - we deal with MHD data
"""
function getImageFromDirectory(dirString,isMHD::Bool, isDicomList::Bool)
#simpleITK object used to read from disk
reader = sitk.ImageSeriesReader()
if(isDicomList)# data in form of folder with dicom files
dicom_names = reader.GetGDCMSeriesFileNames(dirString)
reader.SetFileNames(dicom_names)
return reader.Execute()
elseif(isMHD) #mhd file
return sitk.ReadImage(dirString)
end
end#getPixelDataAndSpacing
"""
becouse Julia arrays is column wise contiguus in memory and open GL expects row wise we need to rotate and flip images
pixels - 3 dimensional array of pixel data
"""
function permuteAndReverse(pixels)
pixels= permutedims(pixels, (3,2,1))
sizz=size(pixels)
for i in 1:sizz[1]
pixels[i,:,:] = reverse(pixels[i,:,:])
end#
for i in 1:sizz[2]
pixels[:,i,:] = reverse(pixels[:,i,:])
end#
return pixels
end#permuteAndReverse
"""
given simple ITK image it reads associated pixel data - and transforms it by permuteAndReverse functions
it will also return voxel spacing associated with the image
"""
function getPixelsAndSpacing(image)
pixelsArr = np.array(sitk.GetArrayViewFromImage(image))# we need numpy in order for pycall to automatically change it into julia array
spacings = image.GetSpacing()
return ( permuteAndReverse(pixelsArr), spacings )
end#getPixelsAndSpacing
Directories - obviously you need to provide path to place where it is stored on your disk. You can download PET/CT data from. You can download example data from https://wwsi365-my.sharepoint.com/:f:/g/personal/s9956jmmswwsiedupl/EstYmEuRHqZNlFIlPBzhbQIBvMwQBJks2lUcCSWgwCYSOg?e=nfW95Q
# directories of PET/CT Data - from https://wiki.cancerimagingarchive.net/display/Public/Head-Neck-PET-CT
dirOfExample ="C:\\GitHub\\JuliaMedPipe\\data\\PETphd\\slicerExp\\all17\\bad17NL-bad17NL\\20150518-PET^1_PET_CT_WholeBody_140_70_Recon (Adult)\\4-CT AC WB 1.5 B30f"
dirOfExamplePET ="C:\\GitHub\\JuliaMedPipe\\data\\PETphd\\slicerExp\\all17\\bad17NL-bad17NL\\20150518-PET^1_PET_CT_WholeBody_140_70_Recon (Adult)\\3-PET WB"
# in most cases dimensions of PET and CT data arrays will be diffrent in order to make possible to display them we need to resample and make dimensions equal
imagePET= getImageFromDirectory(dirOfExamplePET,false,true)
ctImage= getImageFromDirectory(dirOfExample,false,true)
pet_image_resampled = sitk.Resample(imagePET, ctImage)
ctPixels, ctSpacing = getPixelsAndSpacing(ctImage)
# In my case PET data holds 64 bit floats what is unsupported by Opengl
petPixels, petSpacing =getPixelsAndSpacing(pet_image_resampled)
petPixels = Float32.(petPixels)
# we need to pass some metadata about image array size and voxel dimensions to enable proper display
datToScrollDimsB= NuclearMedEye.ForDisplayStructs.DataToScrollDims(imageSize= size(ctPixels) ,voxelSize=ctSpacing, dimensionToScroll = 3 );
# example of texture specification used - we need to describe all arrays we want to display, to see all possible configurations look into TextureSpec struct docs .
textureSpecificationsPETCT = [
TextureSpec{Float32}(
name = "PET",
isNuclearMask=true,
# we point out that we will supply multiple colors
isContinuusMask=true,
#by the number 1 we will reference this data by for example making it visible or not
numb= Int32(1),
colorSet = [RGB(0.0,0.0,0.0),RGB(1.0,1.0,0.0),RGB(1.0,0.5,0.0),RGB(1.0,0.0,0.0) ,RGB(1.0,0.0,0.0)]
#display cutoff all values below 200 will be set 2000 and above 8000 to 8000 but only in display - source array will not be modified
,minAndMaxValue= Float32.([200,8000])
),
TextureSpec{UInt8}(
name = "manualModif",
numb= Int32(2),
color = RGB(0.0,1.0,0.0)
,minAndMaxValue= UInt8.([0,1])
,isEditable = true
),
TextureSpec{Int16}(
name= "CTIm",
numb= Int32(3),
isMainImage = true,
minAndMaxValue= Int16.([0,100]))
];
# We need also to specify how big part of the screen should be occupied by the main image and how much by text fractionOfMainIm= Float32(0.8);
fractionOfMainIm= Float32(0.8);
"""
If we want to display some text we need to pass it as a vector of SimpleLineTextStructs - utility function to achieve this is
textLinesFromStrings() where we pass resies of strings, if we want we can also edit those structs to controll size of text and space to next line look into SimpleLineTextStruct doc
mainLines - will be displayed over all slices
supplLines - will be displayed over each slice where is defined - below just dummy data
"""
import NuclearMedEye.DisplayWords.textLinesFromStrings
mainLines= textLinesFromStrings(["main Line1", "main Line 2"]);
supplLines=map(x-> textLinesFromStrings(["sub Line 1 in $(x)", "sub Line 2 in $(x)"]), 1:size(ctPixels)[3] );
If we want to pass 3 dimensional array of scrollable data we need to supply it via vector ThreeDimRawDat's struct utility function to make creation of those easier is getThreeDims which creates series of ThreeDimRawDat from list of tuples where first entry is String and second entry is 3 dimensional array with data strings needs to be the same as we defined in texture specifications at the bagining data arrays needs to be o the same size and be of the same type we specified in texture specification
import NuclearMedEye.StructsManag.getThreeDims
tupleVect = [("PET",petPixels) ,("CTIm",ctPixels),("manualModif",zeros(UInt8,size(petPixels)) ) ]
slicesDat= getThreeDims(tupleVect )
"""
Holds data necessary to display scrollable data
"""
mainScrollDat = FullScrollableDat(dataToScrollDims =datToScrollDimsB
,dimensionToScroll=1 # what is the dimension of plane we will look into at the beginning for example transverse, coronal ...
,dataToScroll= slicesDat
,mainTextToDisp= mainLines
,sliceTextToDisp=supplLines );
This function prepares all for display; 1000 in the end is responsible for setting window width for more look into SegmentationDisplay.coordinateDisplay
SegmentationDisplay.coordinateDisplay(textureSpecificationsPETCT ,fractionOfMainIm ,datToScrollDimsB ,1000);
As all is ready we can finally display image
Main.SegmentationDisplay.passDataForScrolling(mainScrollDat);
So after invoking this function one should see image sth like below
Interactions
Next all Interactions are done either by mouse or by keyboard shortcuts
left click and drag - will mark active texture (look below - set with alt ...) if it is set to be modifiable in the texture specifications, to the set value and size (by tab...) right click and drag - sets remembered position - when we will change plane of crossection for example from tranverse to coonal this point will be also visible on new plane
all keyboard shortcuts will be activated on RELEASE of keys or by pressing enter while still pressing other; +,- and z keys acts also like enter
shift + number - make mask associated with given number visible
ctrl + number - make mask associated with given number invisible
alt + number - make mask associated with given number active for mouse interaction
tab + number - sets the number that will be used as an input to masks modified by mouse
when tab plus (and then no number) will be pressed it will increase stroke width
when tab minus (and then no number) will be pressed it will increase stroke width
shift + numberA + "-"(minus sign) +numberB - display diffrence between masks associated with numberA and numberB - also it makes automaticall mask A and B invisible
ctrl + numberA + "-"(minus sign) +numberB - stops displaying diffrence between masks associated with numberA and numberB - also it makes automaticall mask A and B visible
space + 1 or 2 or 3 - change the plane of view (transverse, coronal, sagittal)
ctrl + z - undo last action
tab +/- increase or decrease stroke width
F1 - will display wide window for bone Int32(1000),Int32(-1000)
F2 - will display window for soft tissues Int32(400),Int32(-200)
F3 - will display wide window for lung viewing Int32(0),Int32(-1000)
F4, F5 sets minimum (F4) and maximum (KEY_F5) value for display (with combination of + and minus signs - to increase or decrease given treshold) -
In case of continuus colors it will clamp values - so all above max will be equaled to max ; and min if smallert than min
In case of main CT mask - it will controll min shown white and max shown black
In case of maks with single color associated we will step data so if data is outside the rande it will return 0 - so will not affect display F6 - controlls contribution of given mask to the overall image - maximum value is 1 minimum 0 if we have 3 masks and all control contribution is set to 1 and all are visible their corresponding influence to pixel color is 33% if plus is pressed it will increse contribution by 0.1 if minus is pressed it will decrease contribution by 0.1
Benchmark PET/CT
For transparency I include Below code used to benchark PET/CT data
window = Main.SegmentationDisplay.mainActor.actor.mainForDisplayObjects.window
syncActor = Main.SegmentationDisplay.mainActor
using GLFW,DataTypesBasic, ModernGL,Setfield
using BenchmarkTools
BenchmarkTools.DEFAULT_PARAMETERS.samples = 100
BenchmarkTools.DEFAULT_PARAMETERS.seconds =5000
BenchmarkTools.DEFAULT_PARAMETERS.gcsample = true
function toBenchmarkScroll(toSc)
NuclearMedEye.ReactToScroll.reactToScroll(toSc ,syncActor, false)
end
function toBenchmarkPaint(carts)
NuclearMedEye.ReactOnMouseClickAndDrag.reactToMouseDrag(MouseStruct(true,false, carts),syncActor )
end
function toBenchmarkPlaneTranslation(toScroll)
NuclearMedEye.ReactOnKeyboard.processKeysInfo(Option(toScroll),syncActor,KeyboardStruct(),false )
OpenGLDisplayUtils.basicRender(syncActor.actor.mainForDisplayObjects.window)
glFinish()
end
function prepareRAndomCart(randInt)
return [CartesianIndex(12+randInt,13+randInt),CartesianIndex(12+randInt,15+randInt),CartesianIndex(12+randInt,18+randInt),CartesianIndex(2+randInt,10+randInt),CartesianIndex(2+randInt,14+randInt)]
end
#we want some integers but not 0
sc = @benchmarkable toBenchmarkScroll(y) setup=(y = filter(it->it!=0, rand(-5:5,20))[1] )
paint = @benchmarkable toBenchmarkPaint(y) setup=(y = prepareRAndomCart(rand(1:40,1)[1] ))
translations = @benchmarkable toBenchmarkPlaneTranslation(y) setup=(y = setproperties(syncActor.actor.onScrollData.dataToScrollDims, (dimensionToScroll=rand(1:3,2)[1])) )
using BenchmarkPlots, StatsPlots
# Define a parent BenchmarkGroup to contain our suite
scrollingPETCT = run(sc)
mouseInteractionPETCT = run(paint)
translationsPETCT = run(translations)
plot(scrollingPETCT)
If all will be ready you should see sth like on the image below
PURE CT image exaple , MHD file
Files taken from https://sliver07.grand-challenge.org/ As previosly adjust path to your case
exampleLabel = "C:\\GitHub\\JuliaMedPipe\\data\\liverPrimData\\training-labels\\label\\liver-seg002.mhd"
exampleCTscann = "C:\\GitHub\\JuliaMedPipe\\data\\liverPrimData\\training-scans\\scan\\liver-orig002.mhd"
Loading data
imagePureCT= getImageFromDirectory(exampleCTscann,true,false)
imageMask= getImageFromDirectory(exampleLabel,true,false)
ctPixelsPure, ctSpacingPure = getPixelsAndSpacing(imagePureCT)
maskPixels, maskSpacing =getPixelsAndSpacing(imageMask)
We need to pass some metadata about image array size and voxel dimensions to enable proper display
datToScrollDimsB= NuclearMedEye.ForDisplayStructs.DataToScrollDims(imageSize= size(ctPixelsPure) ,voxelSize=ctSpacingPure, dimensionToScroll = 3 );
# example of texture specification used - we need to describe all arrays we want to display
listOfTexturesSpec = [
TextureSpec{UInt8}(
name = "goldStandardLiver",
numb= Int32(1),
color = RGB(1.0,0.0,0.0)
,minAndMaxValue= Int8.([0,1])
),
TextureSpec{UInt8}(
name = "manualModif",
numb= Int32(2),
color = RGB(0.0,1.0,0.0)
,minAndMaxValue= UInt8.([0,1])
,isEditable = true
),
TextureSpec{Int16}(
name= "CTIm",
numb= Int32(3),
isMainImage = true,
minAndMaxValue= Int16.([0,100]))
];
We need also to specify how big part of the screen should be occupied by the main image and how much by text fractionOfMainIm= Float32(0.8);
fractionOfMainIm= Float32(0.8);
"""
If we want to display some text we need to pass it as a vector of SimpleLineTextStructs
"""
import NuclearMedEye.DisplayWords.textLinesFromStrings
mainLines= textLinesFromStrings(["main Line1", "main Line 2"]);
supplLines=map(x-> textLinesFromStrings(["sub Line 1 in $(x)", "sub Line 2 in $(x)"]), 1:size(ctPixelsPure)[3] );
"""
If we want to pass 3 dimensional array of scrollable data"""
import NuclearMedEye.StructsManag.getThreeDims
tupleVect = [("goldStandardLiver",maskPixels) ,("CTIm",ctPixelsPure),("manualModif",zeros(UInt8,size(ctPixelsPure)) ) ]
slicesDat= getThreeDims(tupleVect )
"""
Holds data necessary to display scrollable data
"""
mainScrollDat = FullScrollableDat(dataToScrollDims =datToScrollDimsB
,dimensionToScroll=1 # what is the dimension of plane we will look into at the beginning for example transverse, coronal ...
,dataToScroll= slicesDat
,mainTextToDisp= mainLines
,sliceTextToDisp=supplLines );
This function prepares all for display; 1000 in the end is responsible for setting window width for more look into SegmentationDisplay.coordinateDisplay
SegmentationDisplay.coordinateDisplay(listOfTexturesSpec ,fractionOfMainIm ,datToScrollDimsB ,1000);
As all is ready we can finally display image
Main.SegmentationDisplay.passDataForScrolling(mainScrollDat);
Next part of benchmark for pure CT
#we want some integers but not 0
scPureCt = @benchmarkable toBenchmarkScroll(y) setup=(y = filter(it->it!=0, rand(-5:5,20))[1] )
paintPureCt = @benchmarkable toBenchmarkPaint(y) setup=(y = prepareRAndomCart(rand(1:40,1)[1] ))
translationsPureCt = @benchmarkable toBenchmarkPlaneTranslation(y) setup=(y = setproperties(syncActor.actor.onScrollData.dataToScrollDims, (dimensionToScroll=rand(1:3,2)[1])) )
using BenchmarkPlots, StatsPlots
scB = @benchmarkable toBenchmarkScroll(y) setup=(y = filter(it->it!=0, rand(-5:5,20))[1] )
paintB = @benchmarkable toBenchmarkPaint(y) setup=(y = prepareRAndomCart(rand(1:40,1)[1] ))
translationsB = @benchmarkable toBenchmarkPlaneTranslation(y) setup=(y = setproperties(syncActor.actor.onScrollData.dataToScrollDims, (dimensionToScroll=rand(1:3,2)[1])) )
using BenchmarkPlots, StatsPlots
# Define a parent BenchmarkGroup to contain our suite
scrollingPureCT = run(scB)
mouseInteractionPureCT = run(paintB)
translationsPureCT = run(translationsB)
When all will be ok and you will scroll up you should see sth like below
MedEye3d.SegmentationDisplay
— ModuleMain module controlling displaying segmentations image and data
MedEye3d.SegmentationDisplay.cleanUp
— MethodIn order to properly close displayer we need to : remove buffers that wer use remove shaders remove all textures unsubscibe all of the subscriptions to the mainActor finalize main actor and reinstantiate it close GLFW window
MedEye3d.SegmentationDisplay.coordinateDisplay
— Functioncoordinating displaying - sets needed constants that are storeds in forDisplayConstants; and configures interactions from GLFW events listOfTextSpecs - holds required data needed to initialize textures keeps also references to needed ..Uniforms etc. windowWidth::Int,windowHeight::Int - GLFW window dimensions fractionOfMainIm - how much of width should be taken by the main image heightToWithRatio - needed for proper display of main texture - so it would not be stretched ...
MedEye3d.SegmentationDisplay.passDataForScrolling
— Methodis used to pass into the actor data that will be used for scrolling onScrollData - struct holding between others list of tuples where first is the name of the texture that we provided and second is associated data (3 dimensional array of appropriate type)
MedEye3d.SegmentationDisplay.prepareForDispStruct
— FunctionPreparing ForWordsDispStruct that will be needed for proper displaying of texts numberOfActiveTextUnits - number of textures already used - so we we will know what is still free fragmentshaderwords - reference to fragment shader used to display text vbowords - vertex buffer object used to display words shaderprogram_words - shader program associated with displaying text widthh, heightt - size of the texture - the bigger the higher resolution, but higher computation cost
return prepared for displayStruct
MedEye3d.SegmentationDisplay.registerInteractions
— Methodis using the actor that is instantiated in this module and connects it to GLFW context by invoking appropriate registering functions and passing to it to the main Actor controlling input
MedEye3d.SegmentationDisplay.updateSingleImagesDisplayed
— Methodenables updating just a single slice that is displayed - do not change what will happen after scrolling one need to pass data to actor in listOfDataAndImageNames - struct holding tuples where first entry in tuple is name of texture given in the setup and second is 2 dimensional aray of appropriate type with image data sliceNumber - the number to which we set slice in order to later start scrolling the scroll data from this point
MedEye3d.ReactingToInput.setUpCalcDimsStruct
— Methodadd data needed for proper calculations of mouse, verticies positions ... etc
MedEye3d.ReactingToInput.setUpForScrollData
— Methodadding the data about 3 dimensional arrays that will be source of data used for scrolling behaviour onScroll Data - list of tuples where first is the name of the texture that we provided and second is associated data (3 dimensional array of appropriate type)
MedEye3d.ReactingToInput.setUpMainDisplay
— Methodadding the data into about openGL and GLFW context to enable proper display of main image and masks
MedEye3d.ReactingToInput.setUpWordsDisplay
— Methodadding the data needed for text display; also activates appropriate quad for the display it also configures texture that is build for text display
MedEye3d.ReactingToInput.setUpvalueForMasToSet
— Methodsets value we are setting to the active mask vie mause interaction, in case mask is modifiable
MedEye3d.ReactingToInput.subscribeGLFWtoActor
— Methodwhen GLFW context is ready we need to use this function in order to register GLFW events to Rocket actor - we use subscription for this actor - Roctet actor that holds objects needed for display like window etc... return list of subscriptions so if we will need it we can unsubscribe
MedEye3d.ReactingToInput.updateSingleImagesDisplayedSetUp
— Methodenables updating just a single slice that is displayed - do not change what will happen after scrolling one need to pass data to actor in struct that holds tuple where first entry is -vector of tuples whee first entry in tuple is name of texture given in the setup and second is 2 dimensional aray of appropriate type with image data
- Int - second is Int64 - that is marking the screen number to which we wan to set the actor state
MedEye3d.ReactOnKeyboard.findTextureBasedOnNumb
— Methodgiven number from keyboard input it return array With texture that holds the texture specification we are looking for listOfTextSpecifications - list with all registered Texture specifications numb - string that may represent number - if it does not function will return empty option return Option - either Texture specification or empty Option
MedEye3d.ReactOnKeyboard.parseString
— MethodGiven string it parser it to given object on the basis of with and multiple dispatch futher actions will be done it checks each character weather is numeric - gets substring of all numeric characters and parses it into integer listOfTextSpecifications - list with all registered Texture specifications return option of diffrent type depending on input
MedEye3d.ReactOnKeyboard.reactToKeyboard
— MethodGiven keyInfo struct wit information about pressed keys it can process them to make some actions - generally activating keyboard shortcuts shift + number - make mask associated with given number visible ctrl + number - make mask associated with given number invisible alt + number - make mask associated with given number active for mouse interaction tab + number - sets the number that will be used as an input to masks modified by mouse shift + numberA + "m" +numberB - display diffrence between masks associated with numberA and numberB - also it makes automaticall mask A and B invisible ctrl + numberA + "m" +numberB - stops displaying diffrence between masks associated with numberA and numberB - also it makes automaticall mask A and B visible space + 1 or 2 or 3 - change the plane of view (transverse, coronal, sagittal) ctrl + z - undo last action tab +/- increase or decrease stroke width F1, F2 ... - switch between defined window display characteristics - like min shown white and mx shown black ...
MedEye3d.ReactOnKeyboard.registerKeyboardFunctions
— Methodregistering functions to the GLFW window - GLFW window with Visualization stopListening - atomic boolean enabling unlocking GLFW context
MedEye3d.ReactOnKeyboard.shouldBeExecuted
— Methodreturn true in case the combination of keys should invoke some action
Rocket.on_subscribe!
— Methodwill "tell" what functions should be invoked in order to process keyboard input
MedEye3d.ForDisplayStructs.KeyboardCallbackSubscribable
— Methodgiven pressed keys lik 1-9 and all letters resulting key is encoded as string and will be passed here handler object responsible for capturing action str - name of key lik 1,5 f,.j ... but not ctrl shift etc action - for example key press or release scancode - if key do not have short name like ctrl ... it has scancode
MedEye3d.ReactOnMouseClickAndDrag
— Modulemodule code adapted from https://discourse.julialang.org/t/custom-subject-in-rocket-jl-for-mouse-events-from-glfw/65133/3 it is design to help processing data from -GLFW.SetCursorPosCallback(window, (, x, y) -> println("cursor: x, y")) and for example : cursor: 29.0, 469.0 types Float64 Float64 -GLFW.SetMouseButtonCallback(window, (, button, action, mods) -> println("button action")) for example types MOUSEBUTTON1 PRESS GLFW.MouseButton GLFW.Action The main function is to mark the interaction of the mouse to be saved in appropriate mask and be rendered onto the screen so we modify the data that is the basis of the mouse interaction mask and we pass the data on so appropriate part of the texture would be modified to be displayed on screen
MedEye3d.ReactOnMouseClickAndDrag.addStrokeWidth
— Methodadding the width to the stroke so we will be able to controll how thicly we are painting ...
MedEye3d.ReactOnMouseClickAndDrag.getNewX
— Methodhelper function for translateMouseToTexture
MedEye3d.ReactOnMouseClickAndDrag.getNewY
— Methodhelper function for translateMouseToTexture
MedEye3d.ReactOnMouseClickAndDrag.reactToMouseDrag
— Methodwe use mouse coordinate to modify the texture that is currently active for modifications - we take information about texture currently active for modifications from variables stored in actor from texture specification we take also its id and its properties ...
MedEye3d.ReactOnMouseClickAndDrag.registerMouseClickFunctions
— Methodwe pass coordinate of cursor only when isLeftButtonDown is true and we make it true if left button is presed down - we make it true if the left button is pressed over image and false if mouse get out of the window or we get information about button release imageWidth adn imageHeight are the dimensions of textures that we use to display
MedEye3d.ReactOnMouseClickAndDrag.translateMouseToTexture
— Methodgiven list of cartesian coordinates and some window/ image characteristics - it translates mouse positions to cartesian coordinates of the texture strokeWidth - the property connected to the texture marking how thick should be the brush mouseCoords - list of coordinates of mouse positions while left button remains pressed calcDims - set of values usefull for calculating mouse position return vector of translated cartesian coordinates
MedEye3d.ForDisplayStructs.MouseCallbackSubscribable
— Methodwe define how handler should act on the subject - observable so it will pass event onto subject - here we have 2 events that we want to be ready for - mouse button press example of possible inputs that we would be intrested in for example : cursor: 29.0, 469.0 types Float64 Float64 for example MOUSEBUTTON1 PRESS types GLFW.MouseButton GLFW.Action MOUSEBUTTON1 RELEASE types GLFW.MouseButton GLFW.Action We get two overloads so we will be able to respond with single handler to both mouse click and mouse position Enum GLFW.Action: RELEASE = 0 PRESS = 1 REPEAT = 2 Enum GLFW.MouseButton: MOUSEBUTTON1 = 0 MOUSEBUTTON2 = 1
experiments show that max x,y in window is both 600 if window width and height is 600 so in order to specify weather we are over aour quad we need to know how big is primary quad - defaoul it is occupying 100% of y axis and first left 80% of x axis hence we can calculate max height to equal the height of the window
MedEye3d.ReactToScroll
— Modulemodule that holds functions needed to react to scrolling Generally first we need to pass the GLFW callback to the Rocket obeservable code adapted from https://discourse.julialang.org/t/custom-subject-in-rocket-jl-for-mouse-events-from-glfw/65133/3
MedEye3d.ReactToScroll.reactToScroll
— Functionin case of the scroll p true will be send in case of down - false in response to it it sets new screen int variable and changes displayed screen toBeSavedForBack - just marks weather we wat to save the info how to undo latest action
- false if we invoke it from undoing
MedEye3d.ReactToScroll.registerMouseScrollFunctions
— Methoduploading data to given texture; of given types associated returns subscription in order to enable unsubscribing in the end window - GLFW window stopListening - atomic boolean able to stop the event listening cycle return scrollback - that holds boolean subject (observable) to which we can react by subscribing appropriate actor
Rocket.on_subscribe!
— Methodconfiguting Rocket on Subscribe so we get custom handler of input as we see we still need to define actor
MedEye3d.ForDisplayStructs.ScrollCallbackSubscribable
— Methodwe define how handler should act on the subject - observable so it will pass event onto subject If we will scroll fast number will change much and we will skip some slices
MedEye3d.PrepareWindow.createAndInitShaderProgram
— MethodOn the basis of information from listOfTexturesToCreate it creates specialized shader program
MedEye3d.PrepareWindow.displayAll
— Methodpreparing all for displaying the images and responding to mouse and keyboard input listOfTexturesToCreate- list of texture specifications needed to for example create optimal shader calcDimsStruct - holds important data about verticies, textures dimensions etc.
MedEye3d.TextureManag
— Modulestores functions needed to create bind and update OpenGl textues
MedEye3d.TextureManag.activateTextures
— Methodactivating textures that were already initialized in order to be able to use them with diffrent shader program shader_program- regference to OpenGL program so we will be able to activate textures listOfTextSpecs - list of TextureSpec structs that holds data needed to bind textures to shader program (Hovewer this new shader program have to keep the same ..Uniforms) return unmodified textures
MedEye3d.TextureManag.addTextToTexture
— MethodGiven vector of SimpleLineTextStructs it will return matrix of data that will be used to display text lines - data about text to be displayed calcDimStruct - struct holding important data about size of textures etc. wordsDispObj - object wit needed constants to display text
MedEye3d.TextureManag.assignUniformsAndTypesToMasks
— Methodon the basis of the type supplied in texture characteristic it supplies given set of ..Uniforms to it It would also assign proper openGl types to given julia data type, and pass data from texture specification to opengl context textSpecs - list of texture specificaton that we want to enrich by adding information about ..Uniforms return list of texture specifications enriched by information about ..Uniforms
MedEye3d.TextureManag.createTexture
— Methodcreating texture that is storing values like integer, uint, float values that are representing main image or mask data and which will be used by a shader to draw appropriate colors juliaDataType- data type defined as a Julia datatype of data we are dealing with width,height - dimensions of the texture that we need GL_RType,OpGlType - Open Gl types needed to properly specify the texture they need to be compatible with juliaDataType
MedEye3d.TextureManag.getProperGL_TEXTURE
— Methodassociates GL_TEXTURE UInt32 to given index
MedEye3d.TextureManag.initializeTextures
— Methodinitializing textures - so we basically execute specification for configuration and bind all to OpenGL context listOfTextSpecs - list of TextureSpec structs that holds data needed to calcDimStruct - struct holding necessery data about taxture dimensions, quad propertiess etc.
MedEye3d.TextureManag.setProperOpenGlTypes
— MethodOn the basis of the type associated to texture we set proper open Gl types associated based on https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexImage2D.xhtml and https://www.khronos.org/opengl/wiki/OpenGL_Type
MedEye3d.TextureManag.setuniforms
— Methodhelper for assign..UniformsToMasks On the basis of the name of the Texture it will assign the informs referencs to it
- ..Uniforms for main image will be set separately
MedEye3d.TextureManag.updateImagesDisplayed
— Methodcoordinating updating all of the images, masks... singleSliceDat - holds data we want to use for update forDisplayObjects - stores all needed constants that holds reference to GLFW and OpenGL
MedEye3d.TextureManag.updateTexture
— Methoduploading data to given texture; of given types associated - specified in TextureSpec if we want to update only part of the texture we need to specify offset and size of texture we use Just for reference openGL function definition void glTextureSubImage2D( GLuint texture, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLenum type, const void *pixels);
MedEye3d.DisplayWords
— ModuleModule controlling displaying of the text associated with the segmentation
- either text releted to all slices or just a single one currently displayed or both
MedEye3d.DisplayWords.activateForTextDisp
— MethodIn order to be able to display texture with text we need to activate main shader program and vbo shaderprogram- reference to shader program fragmentshader_words - reference to shader associated with text displaying calcDim - holds necessery constants holding for example window dimensions, texture sizes etc.
MedEye3d.DisplayWords.bindAndActivateForText
— MethodFirst We need to bind fragment shader created to deal with text and supply the vertex shader with data for quad where this text needs to be displayed shader_program- reference to shader program this function is intended to be invoked only once
MedEye3d.DisplayWords.createTextureForWords
— FunctionCreates and initialize texture that will be used for displaying text !!!! important we need to first bind shader program for text display before we will invoke this function numberOfActiveTextUnits - number of textures already used - so we we will know what is still free widthh, heightt - size of the texture - the bigger the higher resolution, but higher computation cost actTextrureNumb -proper OpenGL active texture return fully initialized texture; also it assigne texture to appropriate sampler
MedEye3d.DisplayWords.getTextForCurrentSlice
— Methodwe need to check wether scrolling dat contains some text that can be used for this particular slice display if not we will return only mainTextToDisp
MedEye3d.DisplayWords.reactivateMainObj
— MethodFinally in order to enable later proper display of the images we need to reactivate main quad and shaders shaderprogram- reference to shader program fragmentshader_main- reference to shader associated with main images
MedEye3d.DisplayWords.renderSingleLineOfText
— MethodGiven single SimpleLineTextStruct it will return matrix of data that will be used by addTextToTexture function to display text texLine - source od data textureWidth - available width for a line fontFace - font we use
MedEye3d.DisplayWords.textLinesFromStrings
— Methodutility function that enables creating list of text line structs from list of strings
MedEye3d.Uniforms
— Modulemanaging uniform values - global values in shaders
MedEye3d.Uniforms.changeTextureContribution
— Methodcontrolls contribution of given mask to the overall image - maximum value is 1 minimum 0 if we have 3 masks and all control contribution is set to 1 and all are visible their corresponding influence to pixel color is 33% if plus is pressed it will increse contribution by 0.1 if minus is pressed it will decrease contribution by 0.1 it also modifies given TextureSpec change - how should the texture spec be modified
MedEye3d.Uniforms.coontrolMinMaxUniformVals
— Methodsets minimum and maximum value for display - in case of continuus colors it will clamp values - so all above max will be equaled to max ; and min if smallert than min in case of main CT mask - it will controll min shown white and max shown black in case of maks with single color associated we will step data so if data is outside the rande it will return 0 - so will not affect display
MedEye3d.Uniforms.getuniform
— Method getuniform(program, name) -> GLint
Gets the location of the uniform variable identified by name
.
MedEye3d.Uniforms.setCTWindow
— Methodfunction cotrolling the window for displaying CT scan - min white and max maxshownblack uniformsStore - instantiated object holding references to uniforms controlling displayed window
MedEye3d.Uniforms.setMaskColor
— Methodsets color of the mask
MedEye3d.Uniforms.setTextureVisibility
— Methodsets visibility of the texture
MedEye3d.Uniforms.uniform!
— Function uniform!(location, values...) -> Nothing
Set the uniform vec
variable at location
.
Examples
myuniform = getuniform(program, "someUniform")
# set float vec3
uniform!(myuniform, 1.0, 1.0, 0.5)
# set float vec4
uniform!(myuniform, Cfloat[1, 2, 3, 4])
# set a 4x4 matrix
uniform!(myuniform, rand(Cfloat, 4, 4))
MedEye3d.ShadersAndVerticiesForText.createFragmentShader
— Methodcreating fragment Shader so controlling colors and textures gslString so version of GSLS we are using currently
MedEye3d.ShadersAndVerticiesForText.createVertexShader
— Methodcreating VertexShader so controlling structures like verticies, quads gslString so version of GSLS we are using currently
MedEye3d.ShadersAndVerticiesForText.getWordsVerticies
— Methodgeneretes verticies for quad used for displaying text
MedEye3d.ShadersAndVerticies.createFragmentShader
— Methodcreating fragment Shader so controlling colors and textures gslString so version of GSLS we are using currently
MedEye3d.ShadersAndVerticies.createVertexShader
— Methodcreating VertexShader so controlling structures like verticies, quads gslString so version of GSLS we are using currently
MedEye3d.ShadersAndVerticies.getShaderFileText
— Methodloading the shader from file- so we have better experience writing shader in separate file (can be used if we do not use Custom frag shader)
MedEye3d.OpenGLDisplayUtils.basicRender
— MethodAs most functions will deal with just addind the quad to the screen and swapping buffers
MedEye3d.CustomFragShad
— Modulefunctions that will enable creation of long String that will be the code for custom fragment shader that will be suited for defined textures
MedEye3d.CustomFragShad.addColorUniform
— MethodAdding ..Uniforms resopnsible for colors associated with given mask
MedEye3d.CustomFragShad.addMainTextureStrings
— Methodmanaging main texture and on the basis of the type we will initialize diffrent variables mainTexture - specification of a texture related to main image
MedEye3d.CustomFragShad.addMasksStrings
— MethodAdding string necessary for managing ..Uniforms of masks textures
MedEye3d.CustomFragShad.addSamplerStr
— Methodsetting string representing sampler depending on type
MedEye3d.CustomFragShad.addTypeStr
— Methodgiving variable name associated with given type
MedEye3d.CustomFragShad.addUniformsForNuclearAndSubtr
— MethodAdd ..Uniforms necessary for operation of mask subtraction and for proper display of nuclear medicine masks
MedEye3d.CustomFragShad.addWindowingFunc
— Methodgiving string representing main function defining how windowing of main image should be performed
MedEye3d.CustomFragShad.chooseColorFonuction
— Methodon the basis of texture type gives proper function controlling color of the mask
MedEye3d.CustomFragShad.createCustomFramgentShader
— MethodWe will in couple steps create code for fragment shader that will be based on the texture definitions we gave listOfTexturesToCreate - list of textures on the basis of which we will create custom fragment shader code maskToSubtrastFrom,maskWeAreSubtracting - texture specifications used in order to generate code needed to diplay diffrence between those two masks - every time we want diffrent diffrence we will need to recreate shader invoking this function
MedEye3d.CustomFragShad.divideTexteuresToMainAndRest
— MethodWe divide textures into main image texture and the rest listOfTexturesToCreate - list of textures on the basis of which we will create custom fragment shader code returns tuple where fist entr is the main image texture specification and second is rest of textures
MedEye3d.CustomFragShad.getMasksSubtractionFunction
— Methodused in order to enable subtracting one mask from the other - hence displaying pixels where value of mask a is present but mask b not (order is important) automatically both masks will be set to be invisible and only the diffrence displayed
In order to provide maximum performance and avoid branching inside shader multiple shader programs will be attached and one choosed that will use diffrence needed maskToSubtrastFrom,maskWeAreSubtracting - specifications o textures we are operating on
MedEye3d.CustomFragShad.getNuclearMaskFunctions
— MethodEnable displaying for example nuclear medicine data by applying smoothly changing colors to floating point data 1)in confguration phase we need to have minimum, maximum and range of possible values associated with nuclear mask 2)as uniform we would need to have set of vec4's - those will be used to display colors 3)colors will be set with algorithm presented below a)range will be divided into sections (value ranges) wich numbers will equal length of colors vector - 1 b)in each section the output color will be mix of 2 colors one associated with this section and with next one - contribution of the color associated with given section will vary from 100% at the begining of the section to 0% in the end where 100% of color will be associated with color of next section
MedEye3d.CustomFragShad.initialStrings
— Methodsome initial constants that are the same irrespective of textures
MedEye3d.CustomFragShad.mainFuncString
— Methodcontrolling main function - basically we need to return proper FragColor which represents pixel color in given spot we generete separately r,g and b values by adding contributions from all textures
MedEye3d.CustomFragShad.setMaskInfluence
— MethodGiving value from texture f the texture is set to be visible otherwise 0
MedEye3d.PrepareWindowHelpers
— ModuleIt stores set of functions that need to be composed in order to prepare GLFW window and display verticies needed for texture display
MedEye3d.PrepareWindowHelpers.controllWindowInput
— Methodit will generally be invoked on GLFW.PollEvents() in event loop and now depending on what will be pressed or clicked it will lead to diffrent actions
MedEye3d.PrepareWindowHelpers.createDAtaBuffer
— Methoddata is loaded into a buffer which passes it into thw GPU for futher processing - here the data is just passing the positions of verticies GLSTREAMDRAW the data is set only once and used by the GPU at most a few times. GLSTATICDRAW the data is set only once and used many times. GLDYNAMICDRAW the data is changed a lot and used many times.
MedEye3d.PrepareWindowHelpers.createElementBuffer
— MethodSimilar to the VBO we bind the EBO and copy the indices into the buffer with glBufferData.
MedEye3d.PrepareWindowHelpers.createVertexBuffer
— Methodvertex buffer keeping things simpler
MedEye3d.PrepareWindowHelpers.encodeDataFromDataBuffer
— Methodhow data should be read from data buffer
MedEye3d.PrepareWindowHelpers.glVertexAttribSetting
— Methodshowing how openGL should read data from buffer in GPU in case of code like below it would mean:
first parameter specifies which vertex attribute we want to configure Remember that we specified the location of the position vertex attribute in the vertex shader next argument specifies the size of the vertex attribute. The vertex attribute is a vec2 so it is composed of 2 values. The third argument specifies the type of the data which is GLFLOAT next argument specifies if we want the data to be normalized. If we’re inputting integer data types like int, byte and we’ve set this to GLTRUE The fifth argument is known as the stride and tells us the space between consecutive vertex attributes. Since the next set of position data is located exactly 2 times the size of a float we could’ve also specified the stride as 0 to let OpenGL determine the stride he last parameter is of type void* and thus requires that weird cast. This is the offset of where the position data begins in the buffer.
glVertexAttribPointer positionAttribute, 2, GLFLOAT, false, 0, CNULL
The position data is stored as 32-bit so 4 byte floating point values.
Each position is composed of 2 of those values.
MedEye3d.PrepareWindowHelpers.initializeWindow
— Methodmodified from ModernGL.jl github page and GLFW page stores primary
MedEye3d.StructsManag
— Moduleutilities for dealing data structs like FullScrollableDat or SingleSliceDat
MedEye3d.StructsManag.modifySliceFull!
— Methodmodifies given slice in given coordinates of given data - queried by name data - full data we work on and modify coords - coordinates in a plane of chosen slice to modify (so list of x and y coords) value - value to set for given points return reference to modified slice
MedEye3d.ForDisplayStructs.getLocationDict
— Methodgiven Vector of TextureSpecs it creates dictionary where keys are associated names and values are indicies where they are found in a list
MedEye3d.ForDisplayStructs.ActorWithOpenGlObjects
— TypeActor that is able to store a state to keep needed data for proper display
currentDisplayedSlice::Int=1 # stores information what slice number we are currently displaying mainForDisplayObjects:: forDisplayObjects=forDisplayObjects() # stores objects needed to display using OpenGL and GLFW onScrollData::FullScrollableDat = FullScrollableDat() textureToModifyVec::Vector{TextureSpec}=[] # texture that we want currently to modify - if list is empty it means that we do not intend to modify any texture isSliceChanged::Bool= false # set to true when slice is changed set to false when we start interacting with this slice - thanks to this we know that when we start drawing on one slice and change the slice the line would star a new on new slice textDispObj::ForWordsDispStruct =ForWordsDispStruct()# set of objects and constants needed for text diplay currentlyDispDat::SingleSliceDat =SingleSliceDat() # holds the data displayed or in case of scrollable data view for accessing it calcDimsStruct::CalcDimsStruct=CalcDimsStruct() #data for calculations of necessary constants needed to calculate window size , mouse position ... valueForMasToSet::valueForMasToSetStruct=valueForMasToSetStruct() # value that will be used to set pixels where we would interact with mouse lastRecordedMousePosition::CartesianIndex{3} = CartesianIndex(1,1,1) # last position of the mouse related to right click - usefull to know onto which slice to change when dimensions of scroll change forUndoVector::AbstractArray=[] # holds lambda functions that when invoked will undo last operations maxLengthOfForUndoVector::Int64 = 10 # number controls how many step at maximum we can get back isBusy::Base.Threads.Atomic{Bool}= Threads.Atomic{Bool}(0) # used to indicate by some functions that actor is busy and some interactions should be ceased
MedEye3d.ForDisplayStructs.ForWordsDispStruct
— TypeHolding necessery data to display text - like font related
MedEye3d.ForDisplayStructs.KeyboardCallbackSubscribable
— TypeObject that enables managing input from keyboard - it stores the information also about needed keys wheather they are kept pressed examples of keyboard input (raw GLFW input we process below) action RELEASE GLFW.Action key s StringPRESS key s String action PRESS GLFW.Action key s StringRELEASE key s String action RELEASE GLFW.Action
MedEye3d.ForDisplayStructs.KeyboardStruct
— TypeHolding necessery data to controll keyboard shortcuts
isCtrlPressed::Bool = false# left - scancode 37 right 105 - Int32 isShiftPressed::Bool = false # left - scancode 50 right 62- Int32 isAltPressed::Bool= false# left - scancode 64 right 108- Int32 isEnterPressed::Bool= false# scancode 36 isTAbPressed::Bool= false# isSpacePressed::Bool= false# isF1Pressed::Bool= false isF2Pressed::Bool= false isF3Pressed::Bool= false
lastKeysPressed::Vector{String}=[] # last pressed keys - it listenes to keys only if ctrl/shift or alt is pressed- it clears when we release those case or when we press enter #informations about what triggered sending this particular struct to the actor mostRecentScanCode ::GLFW.Key=GLFW.KEYKP4 mostRecentKeyName ::String="" mostRecentAction ::GLFW.Action= GLFW.RELEASE
MedEye3d.ForDisplayStructs.MainImageUniforms
— TypeHolding references to ..Uniforms used to controll main image
MedEye3d.ForDisplayStructs.Mask
— Typedata needed for definition of mask - data that will be displayed over main image this struct is parametarized by type of 3 dimensional array that will be used to store data
MedEye3d.ForDisplayStructs.MaskTextureUniforms
— Typehold reference numbers that will be used to access and modify given uniform value In order to have easy fast access to the values set the most recent values will also be stored inside In order to improve usability we will also save with what data type this mask is associated for example Int, uint, float etc
MedEye3d.ForDisplayStructs.MouseCallbackSubscribable
— Typestruct that enables reacting to the input from mouse click and drag the input will be Cartesian index represening (x,y) x and y position of the mouse - will be recorded only if left mouse button is pressed or keep presssed
MedEye3d.ForDisplayStructs.MouseStruct
— TypeHolding necessery data to controll mouse interaction
MedEye3d.ForDisplayStructs.ScrollCallbackSubscribable
— Typestruct that enables reacting to the input from scrolling
MedEye3d.ForDisplayStructs.TextureSpec
— TypeHolding the data needed to create and later reference the textures
name::String="" #human readable name by which we can reference texture numb::Int32 =-1 #needed to enable swithing between textures generally convinient when between 0-9; needed only if texture is to be modified by mouse input whichCreated::Int32 =-1 #marks which one this texture was when created - so first in list second ... - needed for convinient accessing ..Uniforms in shaders isMainImage ::Bool = false #true if this texture represents main image isNuclearMask ::Bool = false # used for example in case of nuclear imagiing studies isContinuusMask ::Bool = false # in case of masks if mask is continuus color display we set multiple colors in a vector color::RGB = RGB(0.0,0.0,0.0) #needed in case for the masks in order to establish the range of colors we are intrested in in case of binary mask there is no point to supply more than one color (supply Vector with length = 1) colorSet::Vector{RGB}=[] #set of colors that can be used for mask with continous values strokeWidth::Int32 =Int32(3)#marking how thick should be the line that is left after acting with the mouse ... isEditable::Bool =false #if true we can modify given texture using mouse interaction GLRtype::UInt32 =UInt32(0) #GlRtype - for example GLR8UI or GLR16I OpGlType ::UInt32 =UInt32(0) #open gl type - for example GLUNSIGNEDBYTE or GLSHORT actTextrureNumb ::UInt32 =UInt32(0) #usefull to be able to activate the texture using GLActivetexture - with proper open GL constant associatedActiveNumer ::Int64 =Int64(0) #usefull to be able to activate the texture using GLActivetexture - with proper open GL constant ID::Base.RefValue{UInt32} = Ref(UInt32(0)) #id of Texture isVisible::Bool= true #if false it should be invisible uniforms::TextureUniforms=MaskTextureUniforms()# holds values needed to control ..Uniforms in a shader minAndMaxValue::Vector{T} = []#entry one is minimum possible value for this mask, and second entry is maximum possible value for this mask
MedEye3d.ForDisplayStructs.TextureUniforms
— Typehold reference numbers that will be used to access and modify given uniform value in a shader
MedEye3d.ForDisplayStructs.forDisplayObjects
— TypeDefined in order to hold constant objects needed to display images listOfTextSpecifications::Vector{TextureSpec} = [TextureSpec()] window = [] vertexshader::UInt32 =1 fragmentshader::UInt32=1 shader_program::UInt32=1 stopListening::Base.Threads.Atomic{Bool}= Threads.Atomic{Bool}(0)# enables unlocking GLFW context for futher actions vbo::UInt32 =1 #vertex buffer object id ebo::UInt32 =1 #element buffer object id mainImageUniforms::MainImageUniforms = MainImageUniforms()# struct with references to main image TextureIndexes::Dictionary{String, Int64}=Dictionary{String, Int64}() #gives a way of efficient querying by supplying dictionary where key is a name we are intrested in and a key is index where it is located in our array numIndexes::Dictionary{Int32, Int64} =Dictionary{Int32, Int64}() # a way for fast query using assigned numbers gslsStr::String="" # string giving information about used openg gl gsls version windowControlStruct::WindowControlStruct=WindowControlStruct()# holding data usefull to controll display window
MedEye3d.DataStructs
— Modulestructs helping managing and storing data
MedEye3d.DataStructs.getLocationDict
— Methodgiven Vector of tuples where first is string and second is RawDataToDisp it creates dictionary where keys are those strings - names and values are indicies where they are found
MedEye3d.DataStructs.AnnotationStruct
— Typestruct holding data usefull to controll mouse interaction with image - so stroke width etc.
MedEye3d.DataStructs.CalcDimsStruct
— Typestruct holding data needed for calculating proper mouse position , getting proper size for the texture depending on image dimensions getting into account proportions of diffrent parts of display usefull stats for proper text display
MedEye3d.DataStructs.DataToDisp
— Typehold Data that can be send to be displayed with required metadata
MedEye3d.DataStructs.DataToScrollDims
— Typestores additional data about full dimensions of scrollable dat - this is necessery for switching slicing plane orientation efficiently
MedEye3d.DataStructs.FullScrollableDat
— TypeData that can be displayed and scrolled (so we have multiple slices) struct is mutable becouse in case of the masks data can be changed multiple times and rapidly
MedEye3d.DataStructs.RawDataToDisp
— Typehold raw Data that can be send to be displayed
MedEye3d.DataStructs.SimpleLineTextStruct
— TypeStruct holding line of text with some text metadata
MedEye3d.DataStructs.SingleSliceDat
— TypeData for displaying single slice struct is mutable becouse in case of the masks data can be changed multiple times and rapidly
MedEye3d.DataStructs.ThreeDimRawDat
— Type3 dimensional data for displaying single slice struct is mutable becouse in case of the masks data can be changed multiple times and rapidly
MedEye3d.DataStructs.TwoDimRawDat
— Type2 dimensional ata for displaying single slice struct is mutable becouse in case of the masks data can be changed multiple times and rapidly
MedEye3d.DataStructs.WindowControlStruct
— Typestruct holding data usefull to controll display window
MedEye3d.DataStructs.valueForMasToSetStruct
— Typesimple struct that when passed is giving information about what should be current value we are setting to the mask
MedEye3d.BasicStructs.ConfigurtationStruct
— Typeconfiguration struct that when passed will marks what kind of metrics we are intrested in
MedEye3d.BasicStructs.ImageConstants
— Typeconstants associated with image over which we will evaluate segmentations
MedEye3d.BasicStructs.ResultMetrics
— TypeStruct holding all resulting metrics - if some metric was not calculated its value is just -1
MedEye3d.ModernGlUtil
— Modulecopied from ModernGL github repository
MedEye3d.MaskDiffrence
— Modulefor case when we want to subtract two masks
MedEye3d.MaskDiffrence.displayMaskDiffrence
— MethodSUBTRACTING MASKS used in order to enable subtracting one mask from the other - hence displaying pixels where value of mask a is present but mask b not (order is important) automatically both masks will be set to be invisible and only the diffrence displayed
In order to achieve this we need to have all of the samplers references stored in a list
- we need to set both masks to invisible - it will be done from outside the shader
- we set also from outside uniform marking visibility of diffrence to true
- also from outside we need to set which texture to subtract from which we will achieve this by setting maskAtoSubtr and maskBtoSubtr int ..Uniforms those integers will mark which samplers function will use
- in shader function will be treated as any other mask and will give contribution to output color multiplied by its visibility(0 or 1)
- inside the function color will be defined as multiplication of two colors of mask A and mask B - colors will be acessed similarly to samplers
- color will be returned only if value associated with maskA is greater than mask B and proportional to this difffrence
In order to provide maximum performance and avoid branching inside shader multiple shader programs will be attached and one choosed that will use diffrence needed maskToSubtrastFrom,maskWeAreSubtracting - specifications o textures we are operating on
MedEye3d.MaskDiffrence.processKeysInfo
— Methodfor case when we want to subtract two masks
MedEye3d.MaskDiffrence.undoDiffrence
— Methodfor case we want to undo subtracting two masks
MedEye3d.KeyboardVisibility
— Modulecontrols mask visibility responds to keyboard input
MedEye3d.KeyboardVisibility.processKeysInfo
— Methodprocessing information from keys - the instance of this function will be chosen on the basis mainly of multiple dispatch
MedEye3d.KeyboardVisibility.setVisAndRender
— Methodsets visibility and render the result to the screen
MedEye3d.OtherKeyboardActions
— Modulefunctions to controll stroke width , setting which texture is currently active and actions undoing
MedEye3d.OtherKeyboardActions.processKeysInfo
— Methodin case we want to get new number set for manual modifications toBeSavedForBack - just marks weather we wat to save the info how to undo latest action - false if we invoke it from undoing
MedEye3d.OtherKeyboardActions.processKeysInfo
— Methodwhen tab plus will be pressed it will increase stroke width when tab minus will be pressed it will increase stroke width
MedEye3d.OtherKeyboardActions.processKeysInfoUndo
— MethodIn order to enable undoing last action we just invoke last function from list
MedEye3d.WindowControll
— Modulefunctions that controll window - so basically treshords for mask display
MedEye3d.WindowControll.dispatchToFunctions
— MethodBased on window struct and key info it will controll which function should be invoked
MedEye3d.WindowControll.getNewTresholdChangeValue
— Methodhelper function for setTextureWindow on the basis of given texture spec will give value proportional to the range
MedEye3d.WindowControll.highTreshDown
— Methodsets upper treshold and decrese it
MedEye3d.WindowControll.highTreshUp
— Methodsets upper treshold and Increase it
MedEye3d.WindowControll.lowTreshDown
— Methodsets lower treshold and decrese it
MedEye3d.WindowControll.lowTreshUp
— Methodsets lower treshold and Increase it
MedEye3d.WindowControll.maskContrDown
— Methodsets mask contribution and decrese it
MedEye3d.WindowControll.maskContrUp
— Methodsets mask contribution and increase it
MedEye3d.WindowControll.primaryModificationsOfWindContr
— MethodOn the basis of the input WindowControlStruct and keyInfo it makes necessary primary modifications to WindowControlStruct
MedEye3d.WindowControll.processKeysInfo
— MethodKEYF1 - will display wide window for bone Int32(1000),Int32(-1000) KEYF2 - will display window for soft tissues Int32(400),Int32(-200) KEYF3 - will display wide window for lung viewing Int32(0),Int32(-1000) KEYF4, KEYF5 - sets minimum (F4) and maximum (KEYF5) value for display (with combination of + and minus signs - to increase or decrease given treshold) - in case of continuus colors it will clamp values - so all above max will be equaled to max ; and min if smallert than min in case of main CT mask - it will controll min shown white and max shown black in case of maks with single color associated we will step data so if data is outside the rande it will return 0 - so will not affect display KEY_F6 - controlls contribution of given mask to the overall image - maximum value is 1 minimum 0 if we have 3 masks and all control contribution is set to 1 and all are visible their corresponding influence to pixel color is 33% if plus is pressed it will increse contribution by 0.1 if minus is pressed it will decrease contribution by 0.1
MedEye3d.WindowControll.setTextureWindow
— Methodsets minimum and maximum value for display - in case of continuus colors it will clamp values - so all above max will be equaled to max ; and min if smallert than min in case of main CT mask - it will controll min shown white and max shown black in case of maks with single color associated we will step data so if data is outside the rande it will return 0 - so will not affect display
MedEye3d.WindowControll.setmainWindow
— Methodset main window - min shown white and max shown black on the basis of textur data and windowStruct
MedEye3d.ChangePlane
— Modulecontrols changing plane for example from transverse to saggital ...