readme initial

README.md

@@ -1,6 +1,181 @@
----
-license: other
-license_name: nvidia-open-model-license
-license_link: >-
-  https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
----
+# Asset Harvester | System Model Card
+
+## **Description:**
+
+Asset Harvester is a system that leverages 4 models (see System Architecture below) to generate three-dimensional (3D) assets from a single image or multiple images of vehicles. [Mask2Former](https://docs.google.com/document/d/1OKMAhNruoLE254xLLdIWULPuwUWGNsbpg36BNUnpTSQ/edit?tab=t.0#heading=h.7axn5fq6ipu5)  and [C-RADIO](https://huggingface.co/nvidia/C-RADIO) are used for view extraction from NCore data sessions, the [Multiview Diffusion (Sana-based)](https://docs.google.com/document/d/1y7qU1to8TrV07Tfz3crxJiuA_AL0Wlwwp6C-RW-NoLg/edit?tab=t.0#heading=h.g8ogslbqcx12) is then used to generate 16 multiview images of the input vehicle, and lastly [TokenGS](https://docs.google.com/document/d/1EZWB-had-1MMmrES9bvQlJHpjXQFawR619sX3HvsVpQ/edit?usp=sharing) generates the output 3D asset.
+
+This system is ready for commercial/non-commercial use
+
+### **License/Terms of Use**:
+
+### GOVERNING TERMS: Your use of the model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) .
+
+**Deployment Geography:** Global
+
+## **Automation Level:**
+
+Full Automation
+
+## **Use Case:**
+
+Physical AI developers who are looking to create 3D assets of vehicles for either closed-loop simulation or Synthetic Data Generation (SDG).
+
+## **Known Technical Limitations:**
+
+The system is not guaranteed to perform well with occluded objects or objects that are outside of the common distribution. For example, a heavily occluded vehicle can generate a poor or hallucinated 3D asset, like the following example:
+
+Example of Poor reconstruction of occluded vehicle:
+
+![][image1]
+
+Example of Hallucinated reconstruction of occluded vehicle:
+
+![][image2]
+
+## Known Risk(s):
+
+AV and robotics developers should be aware that this model cannot guarantee a 100% success rate. In cases of unsuccessful generation, the output may not possess an accurate real-world representation of the asset and should not be relied upon in safety-critical simulations.
+
+##
+
+**Release Date:** Public Github \[03/12/2026\]
+
+**Reference(s):** None
+
+## **System Architecture**
+
+**Architecture Diagram:**
+
+![][image3]
+
+The following models are used by this system:
+
+* [Mask2Former Model Card](https://docs.google.com/document/d/1OKMAhNruoLE254xLLdIWULPuwUWGNsbpg36BNUnpTSQ/edit?tab=t.0#heading=h.7axn5fq6ipu5)
+* [C-RADIO Model Card](https://huggingface.co/nvidia/C-RADIO)
+* [Multiview Diffusion (Sana-based) Model Card](https://docs.google.com/document/d/1y7qU1to8TrV07Tfz3crxJiuA_AL0Wlwwp6C-RW-NoLg/edit?tab=t.0#heading=h.g8ogslbqcx12)
+* [TokenGS Model Card](https://docs.google.com/document/d/1EZWB-had-1MMmrES9bvQlJHpjXQFawR619sX3HvsVpQ/edit?usp=sharing)
+
+## **System Input:**
+
+**Input Type(s):** 1 or more images (up until 4\)
+**Input Format:** Red, Green, Blue (RGB)
+**Input Parameters:** Two-Dimensional (2D)
+**Other Properties Related to Input:**
+
+We currently accept up to 4 input images for each object. The resolution of the images are 512x512. The input images are extracted from NVIDIA's NCore data along w/ other metadata needed for downstream processing:
+
+* Camera orientation of each image
+* Camera distance of each image
+* Camera field of view of each image
+* Bounding box dimensions of each object
+
+## **System Output:**
+
+**Output Type(s):** Corresponding 3D Gaussian asset to the object in input images
+**Output Format:** Polygon File Format (PLY)
+**Output Parameters:** Three-Dimensional (3D)
+**Other Properties Related to Output:**
+
+A [PLY file](https://en.wikipedia.org/wiki/PLY_(file_format)#:~:text=PLY%20is%20a%20computer%20file,dimensional%20data%20from%203D%20scanners.) (3D Gaussian Splatting, 3DGS) contains 3D object data with the following specific components:
+
+* **Header**: Defines the file structure, including format (ASCII or binary), Gaussian elements, their properties (e.g., position, appearance coefficients, opacity, scale, rotation), and data types (e.g., float, int).
+* **Gaussian Data**: Stores the parameters of each 3D Gaussian, including its center position (`x, y, z`), and optionally properties such as normals (`nx, ny, nz`), color or spherical harmonics coefficients (`f_dc_0, f_dc_1, f_dc_2`, and higher-order terms), opacity, anisotropic scale, and rotation.
+
+## **Hardware Compatibility:**
+
+**Supported Hardware Microarchitecture Compatibility:**
+
+* NVIDIA Ampere
+* NVIDIA Blackwell
+* NVIDIA Hopper
+* NVIDIA Lovelace
+
+**Preferred/Supported Operating Systems:** Linux
+
+**Hardware Specific Requirements:**
+
+The systems can run on a single GPU with an Nvidia GPU with CUDA Compute Capability greater than or equal to 8.0.   The following is required:
+
+* GPU performance \>= 300 Tflops
+* GPU memory size \>= 30GB
+* GPU memory bandwidth \>= 768 GB/s
+* System RAM \>= 32 GB
+* System disk storage \>= 100GB
+* CPU \>= 16 threads x 3GHz
+
+##
+
+## **System Version:**
+
+Asset\_Harvester\_GA
+
+## **Inference:**
+
+**Engine:** Pytorch
+**Test Hardware:** A100, H100
+
+## **Ethical Considerations:**
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+
+For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards.
+
+Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
+
+## Model Card++
+
+### Bias
+
+| Field | Response |
+| :---- | :---- |
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None |
+| Measures taken to mitigate against unwanted bias: | None |
+
+### Explainability
+
+| Field | Response |
+| :---- | :---- |
+| Intended Domain | Advanced Driver Assistance Systems  |
+| Model Type: | Image-to-3D Asset  |
+| Intended Users: | Autonomous Vehicles developers enhancing and improving  Neural Reconstruction pipelines. |
+| Output | 3D Asset |
+| Describe how the model works | The system takes as an input an image, and outputs a corresponding 3D asset |
+| Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of | None |
+| Technical Limitations | The system is not guaranteed to perform well with occluded objects or objects that are outside of the common distribution. For example, a heavily occluded vehicle can generate a poor or hallucinated 3D asset |
+| Verified to have met prescribed NVIDIA quality standards | Yes |
+| Performance Metrics | PSNR (Peak Signal-to-Noise Ratio) |
+| Potential Known Risks | AV and robotics developers should be aware that this model cannot guarantee a 100% success rate. In cases of unsuccessful generation, the output may not possess an accurate real-world representation of the asset and should not be relied upon in safety-critical simulations. |
+| Licensing | The use of the model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
+
+### Privacy
+
+| Field | Response |
+| :---- | :---- |
+| Generatable or reverse engineerable personal data? | No |
+| Personal data used to create this model? | Yes |
+| Was consent obtained for any personal data used? | Yes |
+| How often is the dataset reviewed? | Before release |
+| Is a mechanism in place to honor data subject right of access or deletion of personal data? | Yes |
+| If personal data was collected for the development of the model, was it collected directly by NVIDIA? | No |
+| If personal data was collected for the development of the model by NVIDIA, do you maintain or have access to disclosures made to data subjects? | Not Applicable |
+| If personal data was collected for the development of this AI model, was it minimized to only what was required? | Yes |
+| Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model? | No |
+| Is there provenance for all datasets used in training? | Yes |
+| Does data labeling (annotation, metadata) comply with privacy laws? | Yes |
+| Is data compliant with data subject requests for data correction or removal, if such a request was made? | Yes |
+| Applicable Privacy Policy | [https://www.nvidia.com/en-us/about-nvidia/privacy-policy/](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) |
+
+### Safety & Security
+
+| Field | Response |
+| :---- | :---- |
+| Model Application(s): | 3D Asset Generation |
+| Describe the life critical impact (if present). | N/A \- The system should not be deployed in a vehicle to perform life-critical tasks. |
+| Use Case Restrictions: | Abide by [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) |
+| Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training |
+
+[image1]: images/image1.png
+
+[image2]: images/image2.png
+
+[image3]: images/image3.png