Goal:

Develop a system that takes a user's input (either a natural language text prompt or a reference image) describing a desired art style and automatically configures the properties of a selected Unity shader to match that style. The output should be a JSON file containing the specific shader property values, compatible with our existing ShaderManager systems in Unity.

Core Challenge:

Our shaders are custom-written (often in HLSL/ShaderLab), lack standardized structures, and the Unity rendering pipeline mapping properties to pixels is inherently non-differentiable using standard methods. The AI needs to bridge the gap between high-level style concepts and low-level, specific shader parameter settings.

Bounty:

$7,500

Key Research Questions

• How can the system extract style-relevant features from an image or text prompt?
• What models or fine-tuning approaches are optimal for mapping visuals to shader semantics?
• How do we map high-level "style" tags to actual shader properties or variants?
• Can we reliably detect and differentiate art styles such as toon, pixel, painterly, cel-shaded, low-poly, etc.?
• What data format should be used to represent shader configuration presets to ensure compatibility with the system?
• How can we validate that the AI's output actually resembles the input style?

Detailed Required Workflow

1. Input: User uploads a reference image or enters an art-style prompt "Ex: Make my game look like a disney pixar movie or User uploads a picture of a comic book page"
2. Analyze input:
   • Extracts stylistic features - classifies art style "Ex: Cartoon animated or Comic book art style)
3. Select:
   • Best matching shader from available shader library. (Based on all our existing pre-compiled shaders)
   • Best variant (if applicable).
   • Utilizes Pre-exported JSON metadata of that shader's properties (if required)
4. Generate:
   • A full JSON preset containing property values that reflect the style.