Help Improve Image Patching/Hole Filling Techniques!

[laistead]

My name is Lesley Istead and I’m a PhD student working under the supervision of Dr. Craig S. Kaplan in the Computer Science Department of the University of Waterloo.

I’m looking for volunteers to help produce/classify images to help understand and improve image patching/hole filling techniques. As a participant, you will be asked to complete one of two simple tasks or complete a survey. The two simple tasks involve drawing a simple shape on a canvas. The survey asks you to choose which image from a set of nearly identical images looks best. You can complete these tasks/surveys as many times as you wish. Each task or survey should take no more than 3 minutes to complete.

This study has been reviewed and received ethics clearance through a University of Waterloo Research Ethics Committee. However, the final decision about participation is yours.

If you are interested in participating, please visit: http://www.csclub.uwaterloo.ca/~lanortha/datagen/start.html

 

[IamSam]

Hello Lesley and welcome to PSG.

I appreciate your patience in giving our mod staff the time to vet your thread. As your keen on feedback (based on comments from your Research Notebook), I will add that it was very difficult to verify who you were and that this study/survey was indeed associated with the University of Waterloo. It took some time and effort but I was able to find out enough about you to reasonably confirm that you and this study seem to be legit. In similar future circumstances, you should provide enough adequate information about yourself and your affiliations to make this task much easier.

As to the actual study page itself, other than your personal introduction and declaration below the study's title, there's no other identification or association to the University. No letterhead or links. The pages URL was the only reference but that can easily be faked. Verification and legitimacy should be made much easier for these types of studies.

Please take the above as constructive. Good luck with your study!

 

[laistead]

Thank you for the feedback! I'll make some changes to the appearance/etc to make it a bit more obvious that this is an official study and not spam

 

[Tom Mann]

It's not obvious exactly what you intend to do with the simple shapes that your interface permits participants to draw, but, in the real world, the issue is almost never the shape of the selection, but rather is the complexity of the underlying image to which a selection will be applied. For example, if you need to remove the contents of a store's window from an image of a city with several buildings, the replacement material must look like something man-made and closely line up with the surroundings (eg, another building), and, for example, your algorithm must not decide to put part of the sky in a near street-level display window.

The bottom line is that I am wondering why your survey doesn't allow survey participants to upload their own images and a selection, (say, in the form of a PSD file), and then see what your algorithms come up with.

Cheers,

Tom M

 

[laistead]

My original thought was to let people upload their own photos and holes/selections from projects they are working on because it would represent the most realistic dataset possible. However, the rights/permissions required to have such a study would have been difficult (not to mention, the ethics clearance that would be required).

That being said, if you have some samples you'd like to contribute, let me know, and we can talk about appropriate permissions. Even discussion about how you use tools like Content Aware Fill/inpainting is very useful to me.

I really appreciate the feedback and discussion

 

[Tom Mann]

Interesting. Thanks for your perspective. FWIW, as a full professor, I supervised the Senior Design course for two departments (as well as taught several graduate level courses) at a major university here in the US for 12 years. I was also often asked to sit in on the reviews in the CS department. FWIW, I just recently retired.

My institution had a major medical school affiliated with it, so my students became well acquainted with IRBs, LOL. We never had any problem with data collection / surveys of the sort you are doing. Perhaps the relevant exemptions up in Canada are not as generous as down here in the States. You might want to check to be sure you don't have an exemption that says that something to the effect that if the data can be collected in such a way that individual participants can't be identified, you don't need to go through the full IRB process. It seems like your current approach would qualify for an exemption on those grounds.

Copyright and permission for the use of images is obviously distinct from IRB approval. This was always handled by having the suppliers of images read and sign a simple limited license agreement form. Even this was only considered as an over abundance of caution if their image was ever going to be used in a publicly accessible document, eg, a scientific / engineering paper. Usually, since only a very limited number of images were ever included with a paper, we only sought such permissions from a very small number of individuals, and only did this as needed. We did not seek licenses for Images that would never see the light of day (eg, that would be analyzed automatically).

Anyway, with respect to technical matters, I obviously don't know the goals and methods you intend to use in your thesis research, but it would seem that unless you are about to commercialize the software you are developing, and hence need lots of real world examples to convince potential sponsors, you could easily make up a huge number of examples yourself simply by pulling images from free (non-copyrighted) image sites and then simply draw in some selections yourself. Select a full human figure in one, a cloud in another, a tree in a third, the exposed skin of the subject in a fourth, the sky in a fifth, etc. etc. -- those parts of images that one would likely want to process separately.

A few examples like this should give you a very good idea of how well your software / algorithms are working. You could also go to the websites of the developers of software such as Vertus' "Fluid Mask", Topaz's masking product, etc. and pull some examples (for your own internal use) from their tutorials. For that matter, a couple of years ago, right on this discussion forum, there were quite a few threads dealing with comparing masking and "cut-out" software, complete with examples. If you can't find any of these, let me know and I'll be glad to dig up some links for you.

Gotta run. More later,

Tom M