One BI system incident lesson

Description: Suddenly, the BI service can not provide services to the external, the entire bi+ reporting system is a oracle11g+obiee11g+weblogic11g+informatica8.6 composition, the entire system is almost a force to complete the line. Yesterday, party a personnel call me, all of a sudden all the BI site, report can not log in, check back-office services, are normal. It is recommended that the other party restart. is still not resolved, so carefully analyze the log. Finally locate the problem.

1. obiee11g sawserver 7001 anaytics web interface login, password user party a confirmed error. Login or real password or user name error. Something's wrong.

2. Try the obiee11g Administrator Tool to connect the online logic model and log in to display the error.

[NQSERROR:37001] Unable to connect to Oracle BI Server instance

3. View the Oracle BI Server Service, OPMNCTL status all services are active. The problem of using the BI Server port can be ruled out, and some of the services in the middleware Bi_platform can crash. Further review the logs for ORACLE BI Server.

Have the following error

[2013-04-18t14:35:17.000+00:00] [Oraclebiservercomponent] [Notification:1] [] [] [ecid:00iejcmvhq4f0fg5xfdciw0000xo000000] [TID:AC8] [85004] MDX Member Name Cache subsystem recovered entries:0, size:0 bytes.
[2013-04-18t14:35:17.000+00:00] [Oraclebiservercomponent] [Error:1] [] [] [ecid:00iejcmvhq4f0fg5xfdciw0000xo000000] [TID:AC8] [13026] Error in getting roles from BI Security Service: ' Er Ror Message from BI Security service:policyset invalid:wsm-06102 policyreference Policy Reference URI "Oracle/wss_username_token_ser Vice_policy "is invalid. ‘
[2013-04-18t14:35:17.000+00:00] [Oraclebiservercomponent] [Notification:1]  [] [] [ecid:00iejcmvhq4f0fg5xfdciw0000xo000000] [Tid:ac8] nqsserver:clustered Oracle BI Server started. version:11.1.1.6.0.120104.0152.000.
[2013-04-18t14:35:17.000+00:00] [Oraclebiservercomponent] [Notification:1] [] [] [ecid:00iejcmvhq4f0fg5xfdciw0000xo000000] [tid:94c] [43071] A connection with Cluster Controller bi-4lna2lrlna7w: 9706 was established.
[2013-04-18t14:35:39.000+00:00] [Oraclebiservercomponent] [Error:1] [] [] [ecid:00iejco8qy8f0fg5xfdciw0000xo000003] [tid:a6c] Error Message from BI Security service:policyset INVALID:WSM -06102 policyreference Policy Reference URI "Oracle/wss_username_token_service_policy" is invalid.
[2013-04-18t14:35:39.000+00:00] [Oraclebiservercomponent] [Error:1] [] [] [ecid:00iejco8qy8f0fg5xfdciw0000xo000003] [tid:a6c] [nqserror:43126] authentication failed:invalid user/passwor D.

4. Can locate the problem, basically is because

This issues occur, when the application wsm-pm are not running. To resolve this, login to EM-application deployments-Internal applications. The applications wsm-pm on AdminServer and managed server should is up and running.

5. Enter WebLogic Em to view the WSM-PM service, which is in a crash state. Reboot. The error is as follows:

RDBMS error,ora-28001:the Password has expired. Suddenly, non-natural disasters are also.

6. Oracle 11G has a new feature to set the password expiration time,

View through SQL

Sql> Conn dev1_mds/admin123
ERROR:
ora-28001:the Password has expired
View through SQL

Sql> SELECT * from Dba_profiles where profile= ' DEFAULT ' and Resource_name= ' PASSW
Ord_life_time '
2;

Profile Resource_name RESOURCE
------------------------------ -------------------------------- --------
LIMIT
----------------------------------------
DEFAULT Password_life_time PASSWORD
360

Change the password for DEV1_MDS
New Password:
Retype the new password:
Password has changed
is connected.
Sql> Conn dev1_biplatform/admin123
ERROR:
ora-28001:the Password has expired


Change the password for Dev1_biplatform
New Password:
Retype the new password:
Password has changed
is connected.

7. Restart the Wms-em service to start successfully.

8. Login BIEE Administrator Tool, login success.

9. Login report and BI system, login successfully.

Summary: This case comparison of the typical, into the big Data era of integrated system problems. The original OLTP or BS system, the problem is relatively single, the bottleneck is relatively single, most of the bottleneck problems concentrated in the database.

Now into the big Data era, is a multi-data source integration (distributed system), Business Unit network integration of the situation, not to say that the database DBA and middleware is not important, is that each link is likely to become a problem environment, to achieve the development of operation and maintenance one mind, it is not easy, pull a bit far, nonsense a bit more. In short, I have a relationship, when the development completed delivery, there is no emphasis on this expired password issue. Recently the problem is inevitable niang.

Ext.: http://www.cnblogs.com/jerryxing/archive/2013/04/18/3028660.html

One BI system incident lesson