Sensors, AFC, Phase I
Development of Polarimetric SWIR Camera System with AI/ML Capabilities to Counter Swarming UAVs
Objective
Develop a polarimetric SWIR camera system with incorporated artificial intelligence and machine learning (AI&ML) capability for enhanced target detection/identification and tracking of swarming UAVs.
Description
To overcome limitations inherent in conventional image-based targeting systems, (e.g., visible, and conventional thermal vision systems) a polarimetrically filtered SWIR camera system based on new high resolution FPA technology is to be developed. [1-3] New SWIR FPAs cost a fraction of the cost (compared to cooled thermal FPAs) and exhibit nearly twice the spatial resolution of their thermal counterparts. Similarly, new SWIR FPA readout technology can produce very large dynamic range resulting in exceptionally low light sensitivity.
To address the highly asymmetric nature of a UAV swarming event, the polarimetric image stream would be analyzed in real-time by an AI&ML algorithm to produce maximum situational awareness. By introducing a polarimetric capability, target imagery is expected to display enhanced information content which can be further exploited by AI/ML analysis.
[4-6] AI&ML algorithm developers should consider recent advances in deep neural networks (DNN) and the maturation of graphical processing unit (GPU) technology optimized for intensive matrix computations. Such AI&ML algorithms are expected to be trained relatively quickly on low-cost GPUs to perform inference on GPUs in real-time. [7-8] Finalized system should be capable of providing appropriate targeting parameters for gimble mounted offensive system to be determined (TBD).
Phase I
During the initial solicitation candidates must identify 1) the optical design proposed for the SWIR polarimetric camera system, and 2) hardware, architecture, and algorithm(s) for the AI&ML operation of the system. As a result, during the Phase I candidates will be expected to conduct a feasibility study which will consist of predictive analysis and/or preliminary prototype development in support of their proposed polarimetric/AI&ML design.
This should include identifying and assessing (with costs) all critical components necessary to develop the proposed system. Specifically, the candidate should define and identify particular focal-plane-array (FPA) architecture, readout circuitry, minimum integration time, optical design, spectral responsivity, and control/analysis hardware and software required for high resolution, high frame-rate operation.
To provide the enhanced spatial and textural detail required for robust targeting, the polarimetric camera system must be capable of producing in real-time a minimum of the following: Stokes imagery, i.e., S0, S1, S2, and a degree-of-linear-polarization (DoLP) image.
[9-10] Analysis should include optical design modeling and optimization in which both radiometric and polarimetric response characteristics are predicted, e.g., noise-equivalent-delta-polarization-state (NEDP). Candidates should strive to achieve a minimum acceptable NEDP of ±1%.
Phase II
Based on the design criteria established during the Phase I, the candidate will procure all necessary components to assemble, test, and demonstrate a fully functional prototype device. Testing will also include evaluation of AI&ML algorithms based on specific test objectives, e.g., percentage of UAVs accurately located/targeted per swarming event and the ability to discern avian clutter from a true threat.
Prototype testing and evaluation will be conducted at a government facility in which optimum functionality will be determined based on range, atmospheric conditions, and tactical scenario. To be conducted concurrent with the prototype development, the contractor will begin identifying all possible commercialization opportunities and partnerships necessary to successfully bring their developed intellectual property (IP) to market.
Phase III
Upon successful completion of Phase II, the contractor may be asked to demonstrate developed AI&ML polarimetric imaging target and tracking system vera the interfacing with identified C-UAV offensive device. Such evaluation will take place at an appropriate U.S. Army field-test facility. This will also include further maturation of the system in which reduction in size, weight, and power (SWaP) will be examined. The candidate is expected to pursue civilian applications and additional commercialization opportunities, e.g., remote sensing of geological formations, enhanced surveillance for homeland/boarder security, detection of buried landmines and IEDs, identification of camouflaged/hidden targets, and night-time facial recognition. [11-14]
Submission Information
Submit in accordance with DoD SBIR BAA 23.2
References:
- Tyo J, Goldstein D, Chenault D, Shaw J. Review of passive imaging polarimetry for remote sensing. Appl Opt. 2006;45(22).
- B. Preece, R. Thompson, V. Hodgkin, K. Gurton, D. Tomkinson, H. Choi, K. Krapels, “Performance Comparison of Conventional IRST (Infrared Search and Track) Sensor versus Polarimetric IRST for the Detection of UAS”, 2014 Military Sensing Symposia (MSS) National Symposium on Sensor & Data Fusion, Springfield, VA. Oct. 28-31, (2014).
- Gurton K.P. Calibrated long-wave infrared (LWIR) thermal and polarimetric imagery of small unmanned aerial vehicles (UAVs) and birds. Army Research Laboratory (US); 2018 Aug. Report No.: ARL-TR-8475.
- Gurton K, Yuffa A, Videen G. Enhanced facial recognition for thermal imagery using polarimetric imaging. Opt. Lett. 2014;39(13):3857–3859.
- Gurton K, Felton M, Pezzaniti L. Remote detection of buried land-mines and IEDs using LWIR polarimetric imaging. Optics Express. 2012; 20:22344–22359.
- Gurton, K and Edmondson, R, “MidIR and LWIR Thermal Polarimetric Imaging Comparison using Receiver Operating Characteristic (ROC) Curve Analysis,” ARL Technical Report, ARL-TR-9092, October 2020
- J. Dai, H. Qi, Y. Xiong, Y. Li et al., “Deformable convolutional networks”, IEEE International Conference on Computer Vision (ICCV), October 22-29 (2017), Venice, Italy.
- K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional neural networks for visual recognition”, IEEE Trans. On Pattern Analysis and Machine Intelligence, 37(9), pp. 1904-1916, (2015).
- Gurton, K., M. A. Felton, R. Mack, C. Farlow, L. Pezzaniti, M. W. Kudenov, D. LeMaster, “MidIR and LWIR polarimetric sensor comparison study”, Proc. SPIE, Polarization: Measurement, Analysis, and Remote Sensing IX, 0277-786, vol. 7672 (2010).
- Duncan L. Hickman, Moira I. Smith, Kyung Su Kim, Hyun-Jin Choi, “Polarimetric imaging: system architectures and trade-offs,” Proc. SPIE 10795, Electro-Optical and Infrared Systems: Technology and Applications XV, 107950B (9 October 2018); doi: 10.1117/12.2325320
- K. Gurton, M. Felton, L. Pezzaniti, “Remote detection of buried land-mines and IEDs using LWIR polarimetric imaging”, Optics Express, Vol. 20 Issue 20, pp.22344-22359 (2012).
- A. Yufa, K. Gurton, G. Videen, “Three-dimensional (3D) facial recognition using passive LWIR polarimetric imaging”, Appl. Opt. vol. 53, no. 36, pp. 8514-8521, Dec. (2014).
- N. Short, S. Hu, P. Gurram, K. Gurton, A. Chan, “Improving cross-modal face recognition using polarimetric imaging”, Optics Letters vol. 40, 6, pp. 882-885 (2015).
- L. Pezzaniti, D. Chenault, K. Gurton, M. Felton, “Detection of obscured targets with IR polarimetric imaging”, Proc. SPIE 9072, Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XIX, 90721D, May 29, (2014).
Objective
Develop a polarimetric SWIR camera system with incorporated artificial intelligence and machine learning (AI&ML) capability for enhanced target detection/identification and tracking of swarming UAVs.
Description
To overcome limitations inherent in conventional image-based targeting systems, (e.g., visible, and conventional thermal vision systems) a polarimetrically filtered SWIR camera system based on new high resolution FPA technology is to be developed. [1-3] New SWIR FPAs cost a fraction of the cost (compared to cooled thermal FPAs) and exhibit nearly twice the spatial resolution of their thermal counterparts. Similarly, new SWIR FPA readout technology can produce very large dynamic range resulting in exceptionally low light sensitivity.
To address the highly asymmetric nature of a UAV swarming event, the polarimetric image stream would be analyzed in real-time by an AI&ML algorithm to produce maximum situational awareness. By introducing a polarimetric capability, target imagery is expected to display enhanced information content which can be further exploited by AI/ML analysis.
[4-6] AI&ML algorithm developers should consider recent advances in deep neural networks (DNN) and the maturation of graphical processing unit (GPU) technology optimized for intensive matrix computations. Such AI&ML algorithms are expected to be trained relatively quickly on low-cost GPUs to perform inference on GPUs in real-time. [7-8] Finalized system should be capable of providing appropriate targeting parameters for gimble mounted offensive system to be determined (TBD).
Phase I
During the initial solicitation candidates must identify 1) the optical design proposed for the SWIR polarimetric camera system, and 2) hardware, architecture, and algorithm(s) for the AI&ML operation of the system. As a result, during the Phase I candidates will be expected to conduct a feasibility study which will consist of predictive analysis and/or preliminary prototype development in support of their proposed polarimetric/AI&ML design.
This should include identifying and assessing (with costs) all critical components necessary to develop the proposed system. Specifically, the candidate should define and identify particular focal-plane-array (FPA) architecture, readout circuitry, minimum integration time, optical design, spectral responsivity, and control/analysis hardware and software required for high resolution, high frame-rate operation.
To provide the enhanced spatial and textural detail required for robust targeting, the polarimetric camera system must be capable of producing in real-time a minimum of the following: Stokes imagery, i.e., S0, S1, S2, and a degree-of-linear-polarization (DoLP) image.
[9-10] Analysis should include optical design modeling and optimization in which both radiometric and polarimetric response characteristics are predicted, e.g., noise-equivalent-delta-polarization-state (NEDP). Candidates should strive to achieve a minimum acceptable NEDP of ±1%.
Phase II
Based on the design criteria established during the Phase I, the candidate will procure all necessary components to assemble, test, and demonstrate a fully functional prototype device. Testing will also include evaluation of AI&ML algorithms based on specific test objectives, e.g., percentage of UAVs accurately located/targeted per swarming event and the ability to discern avian clutter from a true threat.
Prototype testing and evaluation will be conducted at a government facility in which optimum functionality will be determined based on range, atmospheric conditions, and tactical scenario. To be conducted concurrent with the prototype development, the contractor will begin identifying all possible commercialization opportunities and partnerships necessary to successfully bring their developed intellectual property (IP) to market.
Phase III
Upon successful completion of Phase II, the contractor may be asked to demonstrate developed AI&ML polarimetric imaging target and tracking system vera the interfacing with identified C-UAV offensive device. Such evaluation will take place at an appropriate U.S. Army field-test facility. This will also include further maturation of the system in which reduction in size, weight, and power (SWaP) will be examined. The candidate is expected to pursue civilian applications and additional commercialization opportunities, e.g., remote sensing of geological formations, enhanced surveillance for homeland/boarder security, detection of buried landmines and IEDs, identification of camouflaged/hidden targets, and night-time facial recognition. [11-14]
Submission Information
Submit in accordance with DoD SBIR BAA 23.2
References:
- Tyo J, Goldstein D, Chenault D, Shaw J. Review of passive imaging polarimetry for remote sensing. Appl Opt. 2006;45(22).
- B. Preece, R. Thompson, V. Hodgkin, K. Gurton, D. Tomkinson, H. Choi, K. Krapels, “Performance Comparison of Conventional IRST (Infrared Search and Track) Sensor versus Polarimetric IRST for the Detection of UAS”, 2014 Military Sensing Symposia (MSS) National Symposium on Sensor & Data Fusion, Springfield, VA. Oct. 28-31, (2014).
- Gurton K.P. Calibrated long-wave infrared (LWIR) thermal and polarimetric imagery of small unmanned aerial vehicles (UAVs) and birds. Army Research Laboratory (US); 2018 Aug. Report No.: ARL-TR-8475.
- Gurton K, Yuffa A, Videen G. Enhanced facial recognition for thermal imagery using polarimetric imaging. Opt. Lett. 2014;39(13):3857–3859.
- Gurton K, Felton M, Pezzaniti L. Remote detection of buried land-mines and IEDs using LWIR polarimetric imaging. Optics Express. 2012; 20:22344–22359.
- Gurton, K and Edmondson, R, “MidIR and LWIR Thermal Polarimetric Imaging Comparison using Receiver Operating Characteristic (ROC) Curve Analysis,” ARL Technical Report, ARL-TR-9092, October 2020
- J. Dai, H. Qi, Y. Xiong, Y. Li et al., “Deformable convolutional networks”, IEEE International Conference on Computer Vision (ICCV), October 22-29 (2017), Venice, Italy.
- K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional neural networks for visual recognition”, IEEE Trans. On Pattern Analysis and Machine Intelligence, 37(9), pp. 1904-1916, (2015).
- Gurton, K., M. A. Felton, R. Mack, C. Farlow, L. Pezzaniti, M. W. Kudenov, D. LeMaster, “MidIR and LWIR polarimetric sensor comparison study”, Proc. SPIE, Polarization: Measurement, Analysis, and Remote Sensing IX, 0277-786, vol. 7672 (2010).
- Duncan L. Hickman, Moira I. Smith, Kyung Su Kim, Hyun-Jin Choi, “Polarimetric imaging: system architectures and trade-offs,” Proc. SPIE 10795, Electro-Optical and Infrared Systems: Technology and Applications XV, 107950B (9 October 2018); doi: 10.1117/12.2325320
- K. Gurton, M. Felton, L. Pezzaniti, “Remote detection of buried land-mines and IEDs using LWIR polarimetric imaging”, Optics Express, Vol. 20 Issue 20, pp.22344-22359 (2012).
- A. Yufa, K. Gurton, G. Videen, “Three-dimensional (3D) facial recognition using passive LWIR polarimetric imaging”, Appl. Opt. vol. 53, no. 36, pp. 8514-8521, Dec. (2014).
- N. Short, S. Hu, P. Gurram, K. Gurton, A. Chan, “Improving cross-modal face recognition using polarimetric imaging”, Optics Letters vol. 40, 6, pp. 882-885 (2015).
- L. Pezzaniti, D. Chenault, K. Gurton, M. Felton, “Detection of obscured targets with IR polarimetric imaging”, Proc. SPIE 9072, Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XIX, 90721D, May 29, (2014).