8 重庆大学-基于FPGA的实时全高清2D转3D系统

16/6
30/6
Key
Time delayed from scheduled start
Work cpmpleted(on or adead of schedule) Work completed(Behind schedule)
Implementation
Brief experience: Optimizing hardware implementation to
Hui Xie
Bole Li
Implementation
Project timeline
Time 2012.9.1～2012.11.30 2012.12.1～2012.12.31
2013.1.1～2013.6.15 2013.6.16～2013.6.30
Content Algorithms design and soft ware simulation Built the hardware platform and wrote Verilog code
Median filter
Hole-filling
Architecture/Framework
Architecture of the 2D-to-3D conversion system
Ran Liu
Gouqin Tai
• Edge Detection • Depth Estimation
DIBR
M data_valid
Implementation
Brief experience: DDR data timing design.
DDR
Write LE1 Read first frame’s edge data for depth estimation Read second frame’s edge data Read third frame’s edge data Write LE5 Read forth frame’s edge data Read fifth frame’s edge data
Depth Assignment Line map The final depth map
Architecture/Framework
DIBR(Depth Image Based on Rendering)
Depth map
3D image warping Reference 2D image
Weimin Tan
Teamwork
Depth Estimation
• Three-D Image Warping • Median Filter • Hole-filling
• FPGA Design • Simulation & Test
Hardware Implementation
Yingchun Tan
accommodate algorithm effectiveness.(e.g. hole-filling)
Relative parameters hole-filling mark holes M data_valid
detect foreground pixel
fill holes
dilate big holes
Weimin Tan, Yingchun Tan, Bole Li, Hui Xie Chongqing University Supervisor: Ran Liu, Guoqin Tai
Outline
Introduction
Architecture/Framework
Implementation Result
D
Read first frame’s depth data
Read second frame’s depth data
Read third Read forth Depthdepth data frame’s depth data Image Data frame’s Write LD5
D1 D2
Algorithm design of sub-module
1/9
31/10 30/1 18/1 1 2 30/1 18/1 1 2
Software simulation of sub-module algorithm
1/10
Software simulation of 2D-to-3D conversion system
Architecture/Framework
The principle of 2D-to-3D conversion
Generally, 2D-to-3D conversion system mainly contains two step: Depth Generation and DIBR(Depth Image Based on Rendering)
Left View
Original Image
Right View
Architecture/Framework
Depth Estimation
Input image
Color Space Convertion
Gray image
Edge map Line Tracing Edge Detection
E1
E1 E2
Write LE2
E1
Write LE3
LE1
LE1，LE3 ... E1 E1
E2
WriteLE4
E
LE3
E1 E2
Edge ImageE1 Data L5
E
E2 D1
E2
L LE2，2LE4
...
E2
WriteLE6
LE4
E2
LD1，LD3 ...
D1 D2 D1
Write LD3
960 324 11,664 32 128 192 6
22% 24% 23% 31% 6% 1% 16%
Thanks for Your Attention
Write LD1
D2
Write LD2
L 1 LD2，LD4 ... D2te LD4
LD3
D1 D2
LD4
D2
T1
T2
T3
T4
T5
T6
Result
Resource Usage Generated by Xilinx ISE
Device Utilization Summary Slice Logic Utilization Number of Slice Registers Number of Slice LUTs Number of route-thrus Slice Logic Distribution Number of occupied Slices Number of LUT Flip Flop pairs used Number with an unused Flip Flop Number with an unused LUT Number of fully used LUT-FF pairs Number of unique control sets IO Utilization Number of bonded IOBs Specific Feature Utilization Number of BlockRAM/FIFO Total Memory used (KB) Number of BUFG/BUFGCTRLs Number of BUFIOs Number of DSP48Es Number of PLL_ADVs Number of RPM macros Used Available Utilization Note(s) 15,939 207,360 7% 13,662 207,360 6% 798 414,720 1% 6,250 18,842 2,903 5,180 10,759 412 218 78 2,790 10 8 2 1 64 51,840 18,842 18,842 18,842 12% 15% 27% 57%
1/11
The hardware platform building
1/12 16/1 2
31/12
Verilog code of sub-module writing and simulation
1/12
31/1 12/2
Hardware simulation
1/2 12/2
25/6
Producing video and submitting works
Hardware simulation Produced video and submitted works
Implementation
Project timeline
MILESTONE
MONTHS 7
16/8
2012
8
31/8
2013
10 11 12 1 2 3 4 5 6 7
9
Make key project plan
Introduction
Background
With the success of Avatar, 3D applications step into our lives. In a 3D TV system, an adequate supply of 3D content plays an critical role in the early stages. But 3D content generation is a timeconsuming and high cost process at this moment, lack of 3D content is one of mass market barriers. However, there are vast existed 2D material. Thus, 2D-to-3D conversion is proposed to alleviate this problem.