AWS EC2 OEM support

[Dave Herring]

Folks,

(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)

Has anyone set up management agents on AWS EC2 environments to monitor from
an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.

Since we have SSH key pairs set up to reach the AWS servers, my assumption
was I could perform agent installations from OEM (which resides outside of
AWS), using pre-defined Named Credentials that use SSH key pairs.
Unfortunately it seems the connection can't be made that way through OEM,
although I did prove I COULD connect at the OS level using the same method.

I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.

Thx.

[Jeremiah Cetlin Wilton]

I don't understand why on-prem EM13c would regard Oracle databases running on EC2 as any different than any other kind of Oracle database, as long as the networking is set up correctly.

I rarely encounter customers operating anything outside of VPC. But again that should make no difference as long as the correct networking is configured.

Perhaps you could post the EM13c error or log messages when you try to do the remote EM agent installation.

The EM plugin for AWS just lets EM get CloudWatch (generic system metrics) for stuff running in AWS. It's meant to augment, not replace, the use of the EM agent on AWS-based Oracle systems.

Thanks,

Jeremiah


From: "Dave Herring" <***@gmail.com>
To: "Oracle Mailing List" <oracle-***@freelists.org>
Sent: Thursday, July 5, 2018 9:44:04 AM
Subject: AWS EC2 OEM support

Folks,

(I've been given the task of setting up monitoring for a number of Oracle databases on AWS EC2 and unfortunately given little to no guidance, so I apologize upfront if my question seems rather basic.)

Has anyone set up management agents on AWS EC2 environments to monitor from an OEM outside of AWS? We did something similar in the past for RDS environments but I was hoping we wouldn't have to rely on the OEM AWS plugin, which only provides a rather limited subset of functionality of OEM for the envs.

Since we have SSH key pairs set up to reach the AWS servers, my assumption was I could perform agent installations from OEM (which resides outside of AWS), using pre-defined Named Credentials that use SSH key pairs. Unfortunately it seems the connection can't be made that way through OEM, although I did prove I COULD connect at the OS level using the same method.

I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c we'd need to have an Amazon VPC configured and only then could a typical, OEM to agent monitoring configuration and that the only other option is to use the AWS plugin. But, that's just over 1yr old and I wasn't sure if anything has changed since then.

Thx.

[n***@gmail.com]

I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1
2362242.1. If you have internal firewalls this is probably old hat, but if
you don't it's the most likely reason that ssh succeeds but monitoring
doesn't. You'll also need name resolution to be consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my assumption
was I could perform agent installations from OEM (which resides outside of
AWS), using pre-defined Named Credentials that use SSH key pairs.
Unfortunately it seems the connection can't be made that way through OEM,
although I did prove I COULD connect at the OS level using the same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave

[Steve Harville]

Hi Dave

We have this configured with Enterprise Manager 13 and it works fine.
Yes the network needs to be configured correctly.
From the DBA perspective, the main thing is to create an option group on
AWS,
Add the OEM_AGENT option to the option group, then assign your databases to
use the new option group.
See:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Options.OEMAgent.html

Also, Amazon's documentation on how to configure on the Enterprise Manager
side is not good.
Basically when it asks for a host name it really means the RDS "endpoint".

Steve

Post by n***@gmail.com
I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1
2362242.1. If you have internal firewalls this is probably old hat, but if
you don't it's the most likely reason that ssh succeeds but monitoring
doesn't. You'll also need name resolution to be consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my
assumption was I could perform agent installations from OEM (which resides
outside of AWS), using pre-defined Named Credentials that use SSH key
pairs. Unfortunately it seems the connection can't be made that way
through OEM, although I did prove I COULD connect at the OS level using the
same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave

--
Niall Litchfield
Oracle DBA
http://www.orawin.info

[Steve Harville]

Just noticed you are not using RDS.
My response was for RDS only.

Post by Steve Harville
Hi Dave
We have this configured with Enterprise Manager 13 and it works fine.
Yes the network needs to be configured correctly.
From the DBA perspective, the main thing is to create an option group on
AWS,
Add the OEM_AGENT option to the option group, then assign your databases
to use the new option group.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Options.OEMAgent.html
Also, Amazon's documentation on how to configure on the Enterprise Manager
side is not good.
Basically when it asks for a host name it really means the RDS "endpoint".
Steve

Post by n***@gmail.com
I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1
2362242.1. If you have internal firewalls this is probably old hat, but if
you don't it's the most likely reason that ssh succeeds but monitoring
doesn't. You'll also need name resolution to be consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of
Oracle databases on AWS EC2 and unfortunately given little to no guidance,
so I apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my
assumption was I could perform agent installations from OEM (which resides
outside of AWS), using pre-defined Named Credentials that use SSH key
pairs. Unfortunately it seems the connection can't be made that way
through OEM, although I did prove I COULD connect at the OS level using the
same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave

--
Niall Litchfield
Oracle DBA
http://www.orawin.info

[Jeremiah Wilton]

Nevertheless I have forwarded your comments about the RDS emagent docs to the appropriate people. Please let me know if there are any other elements of the docs that you found problematic.

Thanks
Jeremiah

Sent from my iPhone

Post by Steve Harville
Just noticed you are not using RDS.
My response was for RDS only.

Post by Steve Harville
Hi Dave
We have this configured with Enterprise Manager 13 and it works fine.
Yes the network needs to be configured correctly.
From the DBA perspective, the main thing is to create an option group on AWS,
Add the OEM_AGENT option to the option group, then assign your databases to use the new option group.
See: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Options.OEMAgent.html
Also, Amazon's documentation on how to configure on the Enterprise Manager side is not good.
Basically when it asks for a host name it really means the RDS "endpoint".
Steve

I'd imagine that your firewall rules (either virtual or physical or both) will require connectivity between your on-premises OEM and the off-premises EC2 instances on the relevant ports. These are documented in the surprisingly hard to find Note https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1 2362242.1. If you have internal firewalls this is probably old hat, but if you don't it's the most likely reason that ssh succeeds but monitoring doesn't. You'll also need name resolution to be consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of Oracle databases on AWS EC2 and unfortunately given little to no guidance, so I apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor from an OEM outside of AWS? We did something similar in the past for RDS environments but I was hoping we wouldn't have to rely on the OEM AWS plugin, which only provides a rather limited subset of functionality of OEM for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my assumption was I could perform agent installations from OEM (which resides outside of AWS), using pre-defined Named Credentials that use SSH key pairs. Unfortunately it seems the connection can't be made that way through OEM, although I did prove I COULD connect at the OS level using the same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c we'd need to have an Amazon VPC configured and only then could a typical, OEM to agent monitoring configuration and that the only other option is to use the AWS plugin. But, that's just over 1yr old and I wasn't sure if anything has changed since then.
Thx.
--
Dave

--
Niall Litchfield
Oracle DBA
http://www.orawin.info

[Dave Herring]

Yeah, I made the mistake of trusting the FW team when they said they
properly implemented by FW requests. I just checked from our OEM server
that port 3872 and in some cases 1521 are still blocked. I'm currently
checking 4903 from the AWS back to OEM. Unfortunately FW rules are only
pushed Tues and Thurs, even if they made a mistake on something that
already passed.

In the meantime, is it safe to say that outside of adding my public SSH key
to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named
Credential with a credential type of "SSH Key Credentials" should work? I
followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key Named
Credentials " which isn't directly for AWS EC2 but ideally should work.

Dave

Post by n***@gmail.com
I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note https://support.oracle.com/epmos/faces/
DocumentDisplay?id=2362242.1 2362242.1. If you have internal firewalls
this is probably old hat, but if you don't it's the most likely reason that
ssh succeeds but monitoring doesn't. You'll also need name resolution to be
consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my
assumption was I could perform agent installations from OEM (which resides
outside of AWS), using pre-defined Named Credentials that use SSH key
pairs. Unfortunately it seems the connection can't be made that way
through OEM, although I did prove I COULD connect at the OS level using the
same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave

--
Niall Litchfield
Oracle DBA
http://www.orawin.info

[Pete Sharman]

I don’t even remember writing the post that Dave mentioned in his original email, but it sounds like it got sorted out while I was still asleep anyway. 😊



Firewalls are a PITA for EM. I never had to worry about them with the stuff I did at Oracle, but I’ve been going backwards and forwards multiple times with a client recently with the same problem Dave seems to have. I can see why the doc says set it up without firewall rules then add the rules afterwards!



BTW Niall, that support note DOES also point direct to the doc where this stuff is covered - https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632. 😊



Pete



From: oracle-l-***@freelists.org <oracle-l-***@freelists.org> On Behalf Of Dave Herring
Sent: Friday, July 6, 2018 05:21 AM
To: Niall Litchfield <***@gmail.com>
Cc: ORACLE-L <oracle-***@freelists.org>
Subject: Re: AWS EC2 OEM support



Yeah, I made the mistake of trusting the FW team when they said they properly implemented by FW requests. I just checked from our OEM server that port 3872 and in some cases 1521 are still blocked. I'm currently checking 4903 from the AWS back to OEM. Unfortunately FW rules are only pushed Tues and Thurs, even if they made a mistake on something that already passed.



In the meantime, is it safe to say that outside of adding my public SSH key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named Credential with a credential type of "SSH Key Credentials" should work? I followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key Named Credentials " which isn't directly for AWS EC2 but ideally should work.



Dave



On Thu, Jul 5, 2018 at 12:36 PM, <***@gmail.com <mailto:***@gmail.com> > wrote:

I'd imagine that your firewall rules (either virtual or physical or both) will require connectivity between your on-premises OEM and the off-premises EC2 instances on the relevant ports. These are documented in the surprisingly hard to find Note https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1 2362242.1. If you have internal firewalls this is probably old hat, but if you don't it's the most likely reason that ssh succeeds but monitoring doesn't. You'll also need name resolution to be consistent.



On Thu, Jul 5, 2018 at 5:45 PM Dave Herring <***@gmail.com <mailto:***@gmail.com> > wrote:

Folks,



(I've been given the task of setting up monitoring for a number of Oracle databases on AWS EC2 and unfortunately given little to no guidance, so I apologize upfront if my question seems rather basic.)



Has anyone set up management agents on AWS EC2 environments to monitor from an OEM outside of AWS? We did something similar in the past for RDS environments but I was hoping we wouldn't have to rely on the OEM AWS plugin, which only provides a rather limited subset of functionality of OEM for the envs.



Since we have SSH key pairs set up to reach the AWS servers, my assumption was I could perform agent installations from OEM (which resides outside of AWS), using pre-defined Named Credentials that use SSH key pairs. Unfortunately it seems the connection can't be made that way through OEM, although I did prove I COULD connect at the OS level using the same method.



I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c we'd need to have an Amazon VPC configured and only then could a typical, OEM to agent monitoring configuration and that the only other option is to use the AWS plugin. But, that's just over 1yr old and I wasn't sure if anything has changed since then.



Thx.
--
Dave
--
Niall Litchfield
Oracle DBA
http://www.orawin.info

[Ls Cheng]

Hi Pete

Just wondering, why a proxy server
<https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV636>
is requiered (or it is optional?) when there is FW? Isnt it enoguh just
open the ports?

Thanks

Post by Pete Sharman
I don’t even remember writing the post that Dave mentioned in his
original email, but it sounds like it got sorted out while I was still
asleep anyway. 😊
Firewalls are a PITA for EM. I never had to worry about them with the
stuff I did at Oracle, but I’ve been going backwards and forwards
multiple times with a client recently with the same problem Dave seems to
have. I can see why the doc says set it up without firewall rules then add
the rules afterwards!
BTW Niall, that support note DOES also point direct to the doc where this
stuff is covered - https://docs.oracle.com/cd/
E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632.
😊
Pete
Behalf Of *Dave Herring
*Sent:* Friday, July 6, 2018 05:21 AM
*Subject:* Re: AWS EC2 OEM support
Yeah, I made the mistake of trusting the FW team when they said they
properly implemented by FW requests. I just checked from our OEM server
that port 3872 and in some cases 1521 are still blocked. I'm currently
checking 4903 from the AWS back to OEM. Unfortunately FW rules are only
pushed Tues and Thurs, even if they made a mistake on something that
already passed.
In the meantime, is it safe to say that outside of adding my public SSH
key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named
Credential with a credential type of "SSH Key Credentials" should work? I
followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key Named
Credentials " which isn't directly for AWS EC2 but ideally should work.
Dave
I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note https://support.oracle.com/epmos/faces/
DocumentDisplay?id=2362242.1 2362242.1. If you have internal firewalls
this is probably old hat, but if you don't it's the most likely reason that
ssh succeeds but monitoring doesn't. You'll also need name resolution to be
consistent.
Folks,
(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my assumption
was I could perform agent installations from OEM (which resides outside of
AWS), using pre-defined Named Credentials that use SSH key pairs.
Unfortunately it seems the connection can't be made that way through OEM,
although I did prove I COULD connect at the OS level using the same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave
--
Niall Litchfield
Oracle DBA
http://www.orawin.info
--
Dave

[Dave Herring]

Update - the FW team has confirmed our rules were pushed and via splunk
logs they've validated activity over the needed ports. I tried to the push
and install method from OEM but it's initial check comes back with:

2018-07-06_12-21-17:INFO:ssh connect timeout 60000

2018-07-06_12-21-18:INFO:Error Message: PROV-16011: Algorithm negotiation
fail

This matches MOS doc 2373503.1 which says the /etc/ssh/sshd_config files
needed ciphers, both source and dest, yet I've never had to do that before
but then again I've never installed an agent on AWS EC2 before.

Dave

Post by Ls Cheng
Hi Pete
Just wondering, why a proxy server
<https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV636>
is requiered (or it is optional?) when there is FW? Isnt it enoguh just
open the ports?
Thanks
On Fri, Jul 6, 2018 at 12:27 AM, Pete Sharman <

Post by Pete Sharman
I don’t even remember writing the post that Dave mentioned in his
original email, but it sounds like it got sorted out while I was still
asleep anyway. 😊
Firewalls are a PITA for EM. I never had to worry about them with the
stuff I did at Oracle, but I’ve been going backwards and forwards
multiple times with a client recently with the same problem Dave seems to
have. I can see why the doc says set it up without firewall rules then add
the rules afterwards!
BTW Niall, that support note DOES also point direct to the doc where this
stuff is covered - https://docs.oracle.com/cd/E73
210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632. 😊
Pete
Behalf Of *Dave Herring
*Sent:* Friday, July 6, 2018 05:21 AM
*Subject:* Re: AWS EC2 OEM support
Yeah, I made the mistake of trusting the FW team when they said they
properly implemented by FW requests. I just checked from our OEM server
that port 3872 and in some cases 1521 are still blocked. I'm currently
checking 4903 from the AWS back to OEM. Unfortunately FW rules are only
pushed Tues and Thurs, even if they made a mistake on something that
already passed.
In the meantime, is it safe to say that outside of adding my public SSH
key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named
Credential with a credential type of "SSH Key Credentials" should work? I
followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key
Named Credentials " which isn't directly for AWS EC2 but ideally should
work.
Dave
I'd imagine that your firewall rules (either virtual or physical or both)
will require connectivity between your on-premises OEM and the off-premises
EC2 instances on the relevant ports. These are documented in the
surprisingly hard to find Note https://support.oracle.co
m/epmos/faces/DocumentDisplay?id=2362242.1 2362242.1. If you have
internal firewalls this is probably old hat, but if you don't it's the most
likely reason that ssh succeeds but monitoring doesn't. You'll also need
name resolution to be consistent.
Folks,
(I've been given the task of setting up monitoring for a number of Oracle
databases on AWS EC2 and unfortunately given little to no guidance, so I
apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my
assumption was I could perform agent installations from OEM (which resides
outside of AWS), using pre-defined Named Credentials that use SSH key
pairs. Unfortunately it seems the connection can't be made that way
through OEM, although I did prove I COULD connect at the OS level using the
same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave
--
Niall Litchfield
Oracle DBA
http://www.orawin.info
--
Dave

[Jeremiah Cetlin Wilton]

Any sshd messages in the /var/log/auth.log on the DB server at the time of the attempts?

Jeremiah


From: "gdherri" <***@gmail.com>
To: "Ls Cheng" <***@gmail.com>
Cc: "Pete Sharman" <***@westnet.com.au>, "Niall Litchfield" <***@gmail.com>, "Oracle Mailing List" <oracle-***@freelists.org>
Sent: Friday, July 6, 2018 12:17:53 PM
Subject: Re: AWS EC2 OEM support

Update - the FW team has confirmed our rules were pushed and via splunk logs they've validated activity over the needed ports. I tried to the push and install method from OEM but it's initial check comes back with:



2018-07-06_12-21-17:INFO:ssh connect timeout 60000

2018-07-06_12-21-18:INFO:Error Message: PROV-16011: Algorithm negotiation fail

This matches MOS doc 2373503.1 which says the /etc/ssh/sshd_config files needed ciphers, both source and dest, yet I've never had to do that before but then again I've never installed an agent on AWS EC2 before.

Dave

On Fri, Jul 6, 2018 at 4:46 AM, Ls Cheng < [ mailto:***@gmail.com | ***@gmail.com ] > wrote:



Hi Pete

Just wondering, why a [ https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV636 | proxy server ] is requiered (or it is optional?) when there is FW? Isnt it enoguh just open the ports?

Thanks


On Fri, Jul 6, 2018 at 12:27 AM, Pete Sharman < [ mailto:***@westnet.com.au | ***@westnet.com.au ] > wrote:

BQ_BEGIN



I don ’ t even remember writing the post that Dave mentioned in his original email, but it sounds like it got sorted out while I was still asleep anyway. \uD83D\uDE0A



Firewalls are a PITA for EM. I never had to worry about them with the stuff I did at Oracle, but I ’ ve been going backwards and forwards multiple times with a client recently with the same problem Dave seems to have. I can see why the doc says set it up without firewall rules then add the rules afterwards!



BTW Niall, that support note DOES also point direct to the doc where this stuff is covered - [ https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632 | https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632 ] . \uD83D\uDE0A



Pete



From: [ mailto:oracle-l-***@freelists.org | oracle-l-***@freelists.org ] < [ mailto:oracle-l-***@freelists.org | oracle-l-***@freelists.org ] > On Behalf Of Dave Herring
Sent: Friday, July 6, 2018 05:21 AM
To: Niall Litchfield < [ mailto:***@gmail.com | ***@gmail.com ] >
Cc: ORACLE-L < [ mailto:oracle-***@freelists.org | oracle-***@freelists.org ] >
Subject: Re: AWS EC2 OEM support





Yeah, I made the mistake of trusting the FW team when they said they properly implemented by FW requests. I just checked from our OEM server that port 3872 and in some cases 1521 are still blocked. I'm currently checking 4903 from the AWS back to OEM. Unfortunately FW rules are only pushed Tues and Thurs, even if they made a mistake on something that already passed.





In the meantime, is it safe to say that outside of adding my public SSH key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named Credential with a credential type of "SSH Key Credentials" should work? I followed youtube vid " Oracle Enterprise Manager 12c: Create SSH Key Named Credentials " which isn't directly for AWS EC2 but ideally should work.





Dave





On Thu, Jul 5, 2018 at 12:36 PM, < [ mailto:***@gmail.com | ***@gmail.com ] > wrote:
BQ_BEGIN



I'd imagine that your firewall rules (either virtual or physical or both) will require connectivity between your on-premises OEM and the off-premises EC2 instances on the relevant ports. These are documented in the surprisingly hard to find Note [ https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242. | https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242. ] 1 2362242.1. If you have internal firewalls this is probably old hat, but if you don't it's the most likely reason that ssh succeeds but monitoring doesn't. You'll also need name resolution to be consistent.





On Thu, Jul 5, 2018 at 5:45 PM Dave Herring < [ mailto:***@gmail.com | ***@gmail.com ] > wrote:

BQ_BEGIN



Folks,





(I've been given the task of setting up monitoring for a number of Oracle databases on AWS EC2 and unfortunately given little to no guidance, so I apologize upfront if my question seems rather basic.)





Has anyone set up management agents on AWS EC2 environments to monitor from an OEM outside of AWS? We did something similar in the past for RDS environments but I was hoping we wouldn't have to rely on the OEM AWS plugin, which only provides a rather limited subset of functionality of OEM for the envs.





Since we have SSH key pairs set up to reach the AWS servers, my assumption was I could perform agent installations from OEM (which resides outside of AWS), using pre-defined Named Credentials that use SSH key pairs. Unfortunately it seems the connection can't be made that way through OEM, although I did prove I COULD connect at the OS level using the same method.





I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c we'd need to have an Amazon VPC configured and only then could a typical, OEM to agent monitoring configuration and that the only other option is to use the AWS plugin. But, that's just over 1yr old and I wasn't sure if anything has changed since then.





Thx.
--
Dave
--
Niall Litchfield
Oracle DBA
[ http://www.orawin.info/ | http://www.orawin.info ]

BQ_END
--
Dave

BQ_END



BQ_END

[Dave Herring]

It turns out that the ciphers were a problem but in the wrong way - it's
OEM that's out-of-date. The MOS note listed ciphers arcfour, blowfish and
cbc which our sysadmin confirmed are not allowed, security-wise. He also
validated that sshd_config is up-to-date on both ends so the problem seems
to be with OEM (12.1.0.5).

I switched directions and used a "pull" instead of "push" install, grabbing
"AgentPull.sh" using curl on the AWS server and the agent installation
worked fine.

Dave

On Fri, Jul 6, 2018 at 2:46 PM, Jeremiah Cetlin Wilton <

Post by Jeremiah Cetlin Wilton
Any sshd messages in the /var/log/auth.log on the DB server at the time of the attempts?
Jeremiah
------------------------------
*Sent: *Friday, July 6, 2018 12:17:53 PM
*Subject: *Re: AWS EC2 OEM support
Update - the FW team has confirmed our rules were pushed and via splunk
logs they've validated activity over the needed ports. I tried to the push
2018-07-06_12-21-17:INFO:ssh connect timeout 60000
2018-07-06_12-21-18:INFO:Error Message: PROV-16011: Algorithm negotiation fail
This matches MOS doc 2373503.1 which says the /etc/ssh/sshd_config files
needed ciphers, both source and dest, yet I've never had to do that before
but then again I've never installed an agent on AWS EC2 before.
Dave

Post by Ls Cheng
Hi Pete
Just wondering, why a proxy server
<https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV636>
is requiered (or it is optional?) when there is FW? Isnt it enoguh just
open the ports?
Thanks
On Fri, Jul 6, 2018 at 12:27 AM, Pete Sharman <

Post by Pete Sharman
I don’t even remember writing the post that Dave mentioned in his
original email, but it sounds like it got sorted out while I was still
asleep anyway. \uD83D\uDE0A
Firewalls are a PITA for EM. I never had to worry about them with the
stuff I did at Oracle, but I’ve been going backwards and forwards
multiple times with a client recently with the same problem Dave seems to
have. I can see why the doc says set it up without firewall rules then add
the rules afterwards!
BTW Niall, that support note DOES also point direct to the doc where
this stuff is covered - https://docs.oracle.com/cd/
E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632.
\uD83D\uDE0A
Pete
Behalf Of *Dave Herring
*Sent:* Friday, July 6, 2018 05:21 AM
*Subject:* Re: AWS EC2 OEM support
Yeah, I made the mistake of trusting the FW team when they said they
properly implemented by FW requests. I just checked from our OEM server
that port 3872 and in some cases 1521 are still blocked. I'm currently
checking 4903 from the AWS back to OEM. Unfortunately FW rules are only
pushed Tues and Thurs, even if they made a mistake on something that
already passed.
In the meantime, is it safe to say that outside of adding my public SSH
key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named
Credential with a credential type of "SSH Key Credentials" should work? I
followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key
Named Credentials " which isn't directly for AWS EC2 but ideally should
work.
Dave
I'd imagine that your firewall rules (either virtual or physical or
both) will require connectivity between your on-premises OEM and the
off-premises EC2 instances on the relevant ports. These are documented in
the surprisingly hard to find Note https://support.oracle.
com/epmos/faces/DocumentDisplay?id=2362242.1 2362242.1. If you have
internal firewalls this is probably old hat, but if you don't it's the most
likely reason that ssh succeeds but monitoring doesn't. You'll also need
name resolution to be consistent.
Folks,
(I've been given the task of setting up monitoring for a number of
Oracle databases on AWS EC2 and unfortunately given little to no guidance,
so I apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor
from an OEM outside of AWS? We did something similar in the past for RDS
environments but I was hoping we wouldn't have to rely on the OEM AWS
plugin, which only provides a rather limited subset of functionality of OEM
for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my
assumption was I could perform agent installations from OEM (which resides
outside of AWS), using pre-defined Named Credentials that use SSH key
pairs. Unfortunately it seems the connection can't be made that way
through OEM, although I did prove I COULD connect at the OS level using the
same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c
we'd need to have an Amazon VPC configured and only then could a typical,
OEM to agent monitoring configuration and that the only other option is to
use the AWS plugin. But, that's just over 1yr old and I wasn't sure if
anything has changed since then.
Thx.
--
Dave
--
Niall Litchfield
Oracle DBA
http://www.orawin.info
--
Dave

--
Dave

[Rajeev Prabhakar]

Dave,

Have you already tried commenting the
default ciphers entry (at least temporarily
to let the push agent work) in sshd_config
file, restarting the sshd daemon and
giving the push agent a retry ?
Rajeev

Post by Dave Herring
2018-07-06_12-21-17:INFO:ssh connect timeout 60000
2018-07-06_12-21-18:INFO:Error Message: PROV-16011: Algorithm negotiation fail
This matches MOS doc 2373503.1 which says the /etc/ssh/sshd_config files needed ciphers, both source and dest, yet I've never had to do that before but then again I've never installed an agent on AWS EC2 before.
Dave

Post by Ls Cheng
Hi Pete
Just wondering, why a proxy server (https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV636) is requiered (or it is optional?) when there is FW? Isnt it enoguh just open the ports?
Thanks

I don’t even remember writing the post that Dave mentioned in his original email, but it sounds like it got sorted out while I was still asleep anyway. 😊
Firewalls are a PITA for EM. I never had to worry about them with the stuff I did at Oracle, but I’ve been going backwards and forwards multiple times with a client recently with the same problem Dave seems to have. I can see why the doc says set it up without firewall rules then add the rules afterwards!
BTW Niall, that support note DOES also point direct to the doc where this stuff is covered - https://docs.oracle.com/cd/E73210_01/EMADV/GUID-E00C6B3B-D5E2-4E2F-9F94-8A136E3D696E.htm#EMADV632. 😊
Pete
Sent: Friday, July 6, 2018 05:21 AM
Subject: Re: AWS EC2 OEM support
Yeah, I made the mistake of trusting the FW team when they said they properly implemented by FW requests. I just checked from our OEM server that port 3872 and in some cases 1521 are still blocked. I'm currently checking 4903 from the AWS back to OEM. Unfortunately FW rules are only pushed Tues and Thurs, even if they made a mistake on something that already passed.
In the meantime, is it safe to say that outside of adding my public SSH key to the OEM server's $HOME/.ssh/authorized_keys file, then using a Named Credential with a credential type of "SSH Key Credentials" should work? I followed youtube vid "Oracle Enterprise Manager 12c: Create SSH Key Named Credentials " which isn't directly for AWS EC2 but ideally should work.
Dave

I'd imagine that your firewall rules (either virtual or physical or both) will require connectivity between your on-premises OEM and the off-premises EC2 instances on the relevant ports. These are documented in the surprisingly hard to find Note https://support.oracle.com/epmos/faces/DocumentDisplay?id=2362242.1 2362242.1. If you have internal firewalls this is probably old hat, but if you don't it's the most likely reason that ssh succeeds but monitoring doesn't. You'll also need name resolution to be consistent.

Post by Dave Herring
Folks,
(I've been given the task of setting up monitoring for a number of Oracle databases on AWS EC2 and unfortunately given little to no guidance, so I apologize upfront if my question seems rather basic.)
Has anyone set up management agents on AWS EC2 environments to monitor from an OEM outside of AWS? We did something similar in the past for RDS environments but I was hoping we wouldn't have to rely on the OEM AWS plugin, which only provides a rather limited subset of functionality of OEM for the envs.
Since we have SSH key pairs set up to reach the AWS servers, my assumption was I could perform agent installations from OEM (which resides outside of AWS), using pre-defined Named Credentials that use SSH key pairs. Unfortunately it seems the connection can't be made that way through OEM, although I did prove I COULD connect at the OS level using the same method.
I did find a post by Pete Sharman from 5/2016 saying that under OEM 13c we'd need to have an Amazon VPC configured and only then could a typical, OEM to agent monitoring configuration and that the only other option is to use the AWS plugin. But, that's just over 1yr old and I wasn't sure if anything has changed since then.
Thx.
--
Dave

--
Niall Litchfield
Oracle DBA
http://www.orawin.info

--
Dave

--
Dave