Question

In the machine learning process using iDesktopX for example is binary classification, how do we get the confidence score of our inference model?

Answer 1

The confidence score of the inference model can be calculated by the ImageryInference module of iobjectpy, the example code is showing as below:

imagery_inference = ImageryInference(model_path=model_path, gpus=[0], batch_size=1)
imagery_inference.binary_classify_infer(input_data=dom_path, out_data=out_data,
                                        out_dataset_name=out_dataset_name,
                                        offset=128, result_type='grid', confidence_map=True)

After run this, import the output data to iDesktopX, and the confidence score will be shown as the pixel value.

Comments

Hi, so basically, I already try your suggestion through my iDesktopX Python (for your information I already deploy the AI extension and my iDesktopX Python Environment already has iobjectspy in its packages) with this code:

imagery_inference = ImageryInference(model_path="C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Model\saved_model_segformer\saved_model.sdm", gpus=[0], batch_size=1)
imagery_inference.binary_classify_infer(input_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx\DOM_Atas", out_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx",
                                        out_dataset_name='confidence',
                                        offset=128, result_type='grid', confidence_map=True)

The error is:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
NameError: name 'ImageryInference' is not defined

After that, I try following the iObjectpy guide from website (https://iobjectspy.supermap.io/html_en/api/iobjectspy.ml.vision.html?highlight=imageryinference#iobjectspy.ml.vision.ImageryInference.binary_classify_infer). So my code being like this:

imagery_inference = iobjectspy.ml.vision.ImageryInference(model_path="C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Model\saved_model_segformer\saved_model.sdm", gpus=[0], batch_size=1)
imagery_inference.binary_classify_infer(input_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx\DOM_Atas", out_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx",
                                        out_dataset_name='confidence',
                                        offset=128, result_type='grid', confidence_map=True)

Then appears an error like this:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
NameError: name 'iobjectspy' is not defined

So, after that I add some syntax to import iobjectspy:

import iobjectspy

imagery_inference = iobjectspy.ml.vision.ImageryInference(model_path="C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Model\saved_model_segformer\saved_model.sdm", gpus=[0], batch_size=1)
imagery_inference.binary_classify_infer(input_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx\DOM_Atas", out_data="c:\satria\supermap id\opty\big\lintasarta\test geoai\binary classification\image sampling\image_sampling.udbx",
                                        out_dataset_name='confidence',
                                        offset=128, result_type='grid', confidence_map=True)

Then appears an error like this:

Traceback (most recent call last):
  File "<input>", line 3, in <module>
AttributeError: module 'iobjectspy.ml' has no attribute 'vision'

What step must I take to get the confidence score as the pixel value?

---

Hi Satria, to check if you have correctly deployed the python environment, try to run the example script under the path "..\resources_ml\example\example_binary_classification_infer_building.py":
1. Open the script in a python compiler
2. Configure the corresponding python runtime environment
3. change the Line 12 to you own output direction, for example:

out_dir = r"D:\SupermapLearn\AIpy"

4. Run the script, and check whether the script can be successfully executed

If it cannot be executed, please check whether your environment has been configured correctly; 
If it can be executed, please check whether you are correctly reference the corresponding module, the reference syntax should be like this:

from iobjectspy.ml.vision import ImageryInference

---

Hi, thanks for your advice. Here is my script that I use to get confidence map:

import os
import time
import warnings

warnings.filterwarnings('ignore', category=FutureWarning)
from iobjectspy.ml.vision import ImageryInference

#curr_dir = os.path.dirname(os.path.abspath(__file__))
resource_ml_dir = r"C:\Supermap\SuperMap iDesktopX 11i\resources_ml"
data_dir = r"C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling"
model_dir = r"C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Model\saved_model_segformer\saved_model.sdm"
out_dir = r"C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Image_Sampling.udbx"


def example_predict():
    dom_path = r"C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Image_Sample\DOM_Atas\DOM_Atas.tif"
    model_folder = model_dir
    out = out_dir
    out_dataset = 'confidence_building'
    start_time = time.time()
    imagery_inference = ImageryInference(model_path=model_folder, gpus=[0], batch_size=1)
    imagery_inference.binary_classify_infer(input_data=dom_path, out_data=out, out_dataset_name=out_dataset,
                                            offset=0, result_type='grid', confidence_map=True)

    end_time = time.time()
    print('Extraction completed, took a total of {}s, result data saved in {} dataset under {} data source'.format(
        end_time - start_time, out, out_dataset))
    
if __name__ == '__main__':
    example_predict()

Then the result in my iDesktopX python is:

The result of image 'DOM_Atas' have been saved to 'confidence_building'!
Successfully cleared temporary folder:C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\tempSupermapMl_q5jkrxxc
Extraction completed, took a total of 46.60414791107178s, result data saved in C:\Satria\Supermap ID\Opty\BIG\Lintasarta\Test GeoAI\Binary Classification\Image Sampling\Image_Sampling.udbx dataset under confidence_building data source

This is the result of Confidence Map raster:
So finally it can successfully create the confidence map. Thank you.