oralce大学ODI培训资料17

– Flow control then deletes them from flow. – Static control leaves them in datastores. – The ERRORS table can be viewed from Designer or any SQL tool.
17 - 9
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Data Quality Business Rules
• • • • Are defined by designers and business analysts Are stored in the Metadata repository May be applied to application data Are defined in two ways:
•
Reference rules:
– Simple: column A = column B – Complex: column A = function (column B, column C)
•
Validation rules:
– Mandatory columns – Conditions
17 - 11
– Automatically retrieved with other metadata
— —
Rules defined in the databases Obtained by reverse engineering User-defined rules
– Manually added by designers
– It is retained in the sources.
“Data Quality Firewall”:
– – – – Valid data passes through the “firewall.” Invalid data is rejected and stored in error tables. Rejected data can be analyzed or fixed. Fixed data can be processed and/or recycled.
•
Checking data quality after integration is important.
17 - 8
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Business Rules for Data Quality
—
17 - 10
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
From Business Rules to Constraints
• Deduplication rules:
– Primary keys – Alternate keys – Unique indexes
SOURCE APPLICATION INTEGRATION PROCESS TARGET APPLICATION
17 - 5
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Data Quality in Source Applications
– GUI, Storage (DBMS constraints)
Events may create inconsistent data due to:
– Maintenance cycles – Deficient control mechanisms
•
Data may be unsuitable for propagation. Data quality control is, therefore, important before integration.
– By an interface (FLOW and/or STATIC) – By a package (STATIC) – Manually (STATIC)
• • •
Require a Check Knowledge Module (CKM) Are monitored through Operator Copy invalid rows into the ERRORS table
Why Data Quality?
• Data quality is often neglected.
– Development cost – Overconfidence in the actual data
•
Monitoring data quality is critical.
– Low-quality data propagates and contaminates your information system (IS).
17 - 2
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Data Quality
Why and When?
17 - 3
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
•
Flow control (“dynamic data check”):
– Enforces target datastore constraints on data in the flow – Requires an update key defined in the interface – Enables recycling of erroneous data back into the flow
• The IS can be broken into three subsystems:
– Source application(s) – Data integration process(es) – Target application(s)
• •
Data quality should be managed in all the three subsystems. ODI provides the solution for enforcing quality in all the three subsystems.
ERRORS
Static control is started by:
- Interfaces after integration - Packages - Manually
Flow control is started.
CORRECTIONS File
- By interfaces during execution
•
Data integration cannot be separated from data quality.
17 - 4
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
When to Enforce Data Quality
17 - 7
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Data Quality in the Target Applications
• The target applications can be strategic.
Enforcing Data Quality with ODI
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.
Objectives
After completing this lesson, you should be able to: • Describe why and when to enforce data quality • List the different types of data quality business rules that Oracle Data Integrator (ODI) manages • Describe Data Quality System within ODI • Describe how data quality is managed in an interface • Implement flow control during the execution of an interface • Implement static control after the execution of an interface • Enforce constraints and review erroneous records • Manage error table
Data Quality System: Overview
1
Source
ORDERS ERRORS LINES
2
Target
3
Integration process
SALES
Static control is started:
- Automatically (scheduled) - Manually
17 - 14
Copyright © 2010, Oracle and/or its affiliates. All rights reserved.