Copying layer masks into colour channels? Also is this a bug?

[Nanako]

Hi all!
We're using photoshop to prepare assets for a game engine, and i'm trying to find a good workflow, which i can then teach to my artist.
At this point, i have an image with five layers. 12345

Layers 2-5 each have a layer mask on them.

Now what i want to do is, copy the layer mask of layer 2, and paste it into the Red colour channel of layer 1
then mask of layer 3 into the green channel of layer 1
layer 4 into the blue channel
and layer 5 into the alpha channel

So the result is that the channels of that layer will each look identical to the layer masks, and thus layer 1 will look like a wierd (but programatically meaningful) mishmash of colours.

I'm having some problems though.




Apologies for the massive image.
This is the green channel that i've gained from the above procedure. The big gradiented circle in the middle is what i made, but for some reason i have this white stripe down the left side too. I have no idea what that is or why it's there. It appears on both he green and blue channels when i paste in a mask. But if you look at the channel image in the right hand panel, it doesn't seem to have a stripe down it.

the stripe is definitely there, it's affecting the main image, and its annoying me.

any thoughts what might be causing that?

 

[IamSam]

Is it possible to post the .psd file?

 

[Nanako]

Sure, lemme just try to upload it

View attachment EyeColourMaskExample.psd

 

[IamSam]

Without you explaining your exact workflow and walking us through the process one step at a time, my question is...............is the size of the image critical?

If not, could you not simply crop out the white stripe?

Even crop all the way around....


.....white stripe gone.

 

[IamSam]

The base shading layer (red background) is off....


.....but the sclera layer is not ...


Somewhere, the base shading layer was moved.
Have you tried the process from a fresh beginning (new document) to see if the problem reoccurs?

 

[MrToM]

Layer 5 (The very top layer) has a transparent strip down the left hand side.

If you are using Ctrl + Click Thumbnail to create a marquee from that layer then the strip down the left is NOT included as part of the marquee.

Regards.
MrTom.

Scrub that....you found it already.

 

[Nanako]

Have you tried the process from a fresh beginning (new document) to see if the problem reoccurs?

Ahh, i just tried blacking out the base shading layer and then re-copying the channels, that seemed to work. That problem is solved i guess.



Ok now as to my other request, optimisation. My exact workflow currently is:

Alt+left click the vector mask of the layer (sclera to start with)
Press Ctrl+A to select all
Press Ctrl+C to copy it all.
Left click on the top (baseShading) layer
Click on the Channels tab
click on the destination channel (red to start with)
Press Ctrl+V to paste
Click back to the layers tab

Repeat the above steps another three times, once for each other mask, with destinations as green, blue and alpha appropriately.
Is there any way to speed up or automate this process? Perhaps even writing a script? if i could just give her an export script to run it'd be simpler


If you're all wondering the exact point of all this, we're making a game with user customisation via tinting textures. I've written a custom shader for unity that takes a colour mask image, where the masks for tinting of the texture are encoded in the RGBA channels of the color mask image.

The reason we have it in layer masks to begin with, is for design and previewing purposes. it allows us to see roughly what the final result will look like ingame, without having to export and test it for every tiny change, and provides a visual environment for carefully adjusting how the channels blend.. What i need help with now is just making this exporting process simpler and quicker.

 

[thebestcpu]

Hi Nanako
I think this can be done as an Action with the steps I outline below.
To be honest, it is a little confusing to me as I usually number Layers from the botton up and apparently you do so from the top down.
I also don't know if you want to preserve all the Layers or not.
Also, using the nomenclature of "alpha" mask is used a lot of different ways so you last step of using the bottom Layer mask for the alpha channel is not clear to me (do you mean the transparency channel embedded with the pixel Layer?

So let me get you started. I will assume if you want to preserve any Layer then you will add that to my steps below. I will focus on only creating the combined Layer using the Layer Masks (not Vector Mask) as channels.
So I am ignoring the Baseshading Layer and Pupil Layer for now in this post

- First make sure any selection is deleted/turned off
- For the sclera, outer iris, and inner iris layers fill the pixels with pure black

- For Inner Iris Layer with Layer Mask right click and "Add Mask to Selection"
- Fill Layer with Blue
- Delete Selection
- Delete Layer Mask


- For Outer Iris Layer with Layer Mask right click and "Add Mask to Selection"
- Fill Layer with Green
- Delete Selection
- Delete Layer Mask


- For Sclera Layer with Layer Mask right click and "Add Mask to Selection"
- Fill Layer with Red
- Delete Selection
- Delete Layer Mask

- Change Blending mode of Sclera, and Outer Iris Layer to Linear Dodge (adds the Layers)
- Merge those 3 Layers and you should have a Layer with the RGB channels matching the Layer Masks as you specified.

After the above, it is simple to change the blending as desired of adding the last Layer or other steps. Just was not sure what you wanted.

The above approach (if I wrote the instructions correctly) is basically making each Layer its own independent R,G,B Layer with values per the Layer Mask and then adds up these independent channels. I am pretty sure this can all be done in an Action
Hope this gets you going.

BTW -