If I want to make my own project dataset, how can I get this pkl file.

[xyysname]

real_test\cat\000001-coor.pkl.
Hello.What does the content of this pkl file mean? If I want to make my own project dataset, how can I get this pkl file.Can I train the model without this pkl file? What impact will it have?

[hirohitokato]

I also would like to know how to create the XXX-coor.pkl file, too.

[chyphen7]

I would also be interested in knowing how to train our own data to create *-coor.pkl. Thanks.

[LZGMatrix]

@xyysname @hirohitokato @chyphen7 Thanks for your questions. These "*-coor.pkl" coordinates files are the ground-truth coordinates labels to train the rotation head of our CDPN model. They can be created by the 3D object models and the pose annotations provided by the LM/LMO dataset. Concreately, you can simply project the 3D object models to the image plane using the ground-truth pose annotation to create them.

[hirohitokato]

@LZGMatrix Thank you so much for a comment!

Do you use/create the tool for building the "*-coor..pkl" files?
If so, I want to use it so that we can train your model with the best performance.

[xyysname]

@LZGMatrix Thanks for your answer, but I want to confirm whether the specific production steps I now know are correct.
Now,I don't have depth pictures in my hand, only the ply file (point cloud coordinates) and pose information of the model. Can I directly use the 3D point cloud coordinates of the model, that is, the point cloud coordinates in the ply file, to create the coor in the pkl file? Is the "coor" in the plk file obtained by using the pose matrix to transform the point cloud coordinates in the ply file?

[xyysname]

我现在制作数据集遭遇了制作这个 pkl文件里面the ground-truth coordinates 的困难。这个coor坐标的含义是物体像素点对应的真实的3d坐标，那我怎么能够得到这个真实的3d坐标呢，具体要使用哪些工具呢？我现在手里不能得到深度图片，只有rgb图片和图片对应的pose矩阵以及物体的ply点云文件，那我能用这些东西把那个ground-truth coordinates制作出来吗？

[hirohitokato]

I am trying to create a -coor.pkl file, but I still can't get it to work. I would like to know what tool or scripts you used to convert it.

[petrock99]

I too am interested in setting up my own dataset and am getting hung up by the pkl files. I see that there are .ply & .obj files in root/dataset/lm/models. If would be great if a method to convert one of the 3d model files to a .pkl file were made available. I think that would answer a lot of questions. Thanks.

[petrock99]

@LZGMatrix, correct me if I'm wrong, but I think I was able to reverse engineer the image portion of the pkl file. After some normalizing of the data it appears to be a 2D rendering of the corresponding 3D model with the matching pose.txt rotations applied, as @LZGMatrix mentioned above. Each raw channel value is roughly between -0.05 & 0.05. When normalized to between 0 & 1, to make matplotlib happy, each pixel has a different color of the rainbow.

@LZGMatrix:

• Does the different color per pixel & -0.05 to 0.05 value range matter to the algorithm?
• Is it essentially a unit vector field?
• Should I file a separate 'Issue' to request an official method to generate .pkl files from the .ply/.obj & pose.txt files or would you like to use this one?

I've written a small script to convert the .pkl files to .pngs in case anyone is interested.

import pickle as pkl
from matplotlib import image

def norm(coor):    # Normalize between 0.0 & 1.0
    return ((coor - coor.min()) * 255. / (coor.max() - coor.min())) / 255.

with open("path/to/coor.pkl", "rb") as f:
    image.imsave('coor.png', norm(pkl.load(f)['coor']))

[lululovecode]

我现在制作数据集遭遇了制作这个 pkl文件里面the ground-truth coordinates 的困难。这个coor坐标的含义是物体像素点对应的真实的3d坐标，那我怎么能够得到这个真实的3d坐标呢，具体要使用哪些工具呢？我现在手里不能得到深度图片，只有rgb图片和图片对应的pose矩阵以及物体的ply点云文件，那我能用这些东西把那个ground-truth coordinates制作出来吗？

我也遇到了一样的问题，怎么制作生成pkl文件呢？不然一直无法训练自己的数据集....

[LZGMatrix]

@LZGMatrix Thank you so much for a comment!

Do you use/create the tool for building the "*-coor..pkl" files? If so, I want to use it so that we can train your model with the best performance.

Hi, as said before, these *-coor.pkl are generated by projecting the 3d object model (i.e. *.ply file) using the ground-truth pose annotations. It can be implemented with no more than 100 lines of code.

[LZGMatrix]

@hirohitokato @lululovecode @xyysname @chyphen7 Hi, as said before, these *-coor.pkl are generated by projecting the 3d object model (i.e. *.ply file) using the ground-truth pose annotations. It can be implemented with no more than 100 lines of code. If all people still cannot achive it, maybe in the near future (when I have spare time), I will provide a short technical article to illustrate it. Anyway, it is a very simple and straightforward thing.

[BoBoINVICTUS]

Thanks for your questions. These "*-coor.pkl" coordinates files are the ground-truth coordinates labels to train the rotation head of our CDPN model. They can be created by the 3D object models and the pose annotations provided by the LM/LMO dataset. Concreately, you can simply project the 3D object models to the image plane using the ground-truth pose annotation to create them.

Hello，has this technical article been published?

[LZGMatrix]

@BoBoINVIDICUS

Hi, I will release it in several days, hold on~