Resource Group: i2-redcap-monitor

Overview

The i2-redcap-monitor resource group is designed to support various services, including data collection, event logging, monitoring, and alerts within an Azure environment. It primarily utilizes Azure Event Hubs for data ingestion, along with Monitoring and Insights services to ensure system performance and security. This environment is intended to be used for applications that require robust data connection and management, facilitating analytics and operational insights.

Resources

1. Event Hub Namespace: i2db-redcapprod

   • Type: Microsoft.EventHub/namespaces
   • Location: Central US
   • Relationship: Serves as a hub for streaming data. It has event hubs configured for various logging purposes.
   • Configurations:
     • Geo-Replication: Enabled, with location as Central US.
     • Public Network Access: Enabled, allowing external access.
     • Zone Redundant: True, ensuring availability during failures.

2. Action Group: azure_infra_Group

   • Type: microsoft.insights/actionGroups
   • Location: Global
   • Relationship: This action group is triggered by alerts from different monitored resources.
   • Configurations:
     • Email Notifications: Sends alerts to alvinn@wustl.edu.

3. Data Collection Rule: MSVMI-RedCap-Dev_VM

   • Type: Microsoft.Insights/dataCollectionRules
   • Location: Central US
   • Purpose: Collects telemetry data from Virtual Machines and aggregates it to specified destinations.
   • Relationships:
     • Log Analytics Workspace: It collects data to IcsrRdcapResourcesWorkspcae.

4. Private Endpoint: I2db-prod-plunk

   • Type: Microsoft.Network/privateEndpoints
   • Location: Central US
   • IP Address: 10.24.96.154
   • Purpose: Establishes a private link connection to the i2db-rdcprod-splunk Event Hub, restricting network access for enhanced security.
   • Subnet: Connected to the subnet named main within the i2-redcap_main_vnet.

5. Event Hubs (Multiple)

   • Names:
     • i2db-appgw-logs
     • i2db-app-svc-logs
     • i2db-database-logs
     • i2db-vmnic-logs
   • Type: Microsoft.EventHub/namespaces/eventhubs
   • Purpose: To handle various logs and data inputs from different applications/services. Each hub is configured with retention policies to manage log lifetimes.
   • Configurations:
     • Message Retention: Varies by event hub, typically from 1 hour to 7 hours.

6. Network Rulesets

   • Event Hub Namespace: i2db-rdcapprod
   • Default Action: Allow for public access.
   • Security: Has an associated network ruleset for managing access and connections.

7. Activity Log Alerts: Redcap_MySQl_Admin_Actions

   • Type: microsoft.insights/activityLogAlerts
   • Location: Global
   • Purpose: Monitors and alerts on administrative actions in MySQL flexible servers.
   • Relationships: Monitors various MySQL flexible servers defined in the parameters.

8. Metric Alerts: Redcap_VM_CPU_Utilization

   • Type: microsoft.insights/metricAlerts
   • Location: Global
   • Purpose: Monitors CPU utilization across various VMs.
   • Alert Criteria: Triggered if CPU exceeds 60%.

Data Storage

Data within this resource group is primarily stored in Azure Event Hubs, which act as a centralized service for data ingestion. Each Event Hub corresponds to different log data types, maintaining lean and effective management of telemetry and operational data. Real-time data can flow from various sources (VMs, services, etc.) into these hubs for further processing and analysis, ensuring a structured data collection mechanism.

Networking

The i2-redcap-main-vnet virtual network is utilized within this resource model. Here are some key configurations:

• Subnet: There is a subnet named main which hosts the private endpoint I2db-prod-plunk.
• Private Endpoint: Ensures that resources can communicate with the i2db-rdcprod-splunk Event Hub securely through the private IP 10.24.96.154.
• Public Network Access: Some Event Hubs have public access enabled, which must be cautiously managed to prevent unauthorized access.

Security Overview

Security is a critical aspect of the configurations in this resource group. Below are some observations and recommendations:

• Public Network Access: Some Event Hubs are publicly accessible. Ensure that access control policies are strictly enforced to limit access and avoid unwanted data exposure.
• Private Endpoints: Utilization of private endpoints is a good practice as it restricts public access and enables secure connectivity to resources over a private link.
• Network Rulesets: Consider restricting access to certain IP ranges rather than allowing all public access, especially for resources involving sensitive data.
• Monitoring Alerts: Set up extensive alerting around any changes to the access control policies of the Event Hubs and private endpoints.

Other Information

• Cost Management: Monitor usage of Event Hubs and associated services to maintain cost efficiency. Azure Event Hubs typically incurs charges based on throughput, data retention, and ingress/egress data operations.
• Scalability: Ensure that Event Hubs and Log Analytics resources are scaled appropriately based on expected traffic, application workloads, or telemetry volume.
• Future Considerations: Depending on system growth and data usage, consider implementing additional metrics and logging capabilities to improve incident response and maintain a proactive stance toward performance monitoring.