Splunk

Role: You are a Splunk search engineer. Task: produce a single, optimized SPL query that returns the top 10 error types from application logs in the last 24 hours. Constraints: use index=app_logs, earliest=-24h@h latest=now, group by error_message and error_code, include absolute counts and percent of total, sort by count desc, avoid expensive subsearches or joins. Output format: return only the SPL query on the first line and one concise (<=20 words) explanation line after it. Example fields available: _time, host, sourcetype, error_message, error_code.

Role: You are a Splunk alert author. Task: craft a scheduled alert (SPL + alert configuration) that detects sustained CPU usage spikes across hosts. Constraints: use metric or event index metric_cpu or index=infra_metrics, evaluate every 5 minutes, trigger when avg CPU% > 85% for at least 10 minutes, group by host, return host, avg_cpu, last_seen. Output format: present the SPL query, suggested cron schedule, threshold type (per-host), trigger condition text, and suggested severity tag. Example: show how to avoid noisy single-sample spikes by using moving average or windowed aggregation.

Role: You are a Splunk observability engineer. Task: produce a structured dashboard plan for a service SLO with 3 panels: (1) SLI error rate over 30 days, (2) SLO burn rate and error budget remaining, (3) recent incidents impacting SLO. Constraints: accept variable 'service_name', compute SLI as successful_requests/total_requests, use earliest=-30d latest=now, show threshold colors (green/yellow/red). Output format: JSON array with each panel object containing title, visualization type, exact SPL query (using index=apm or index=app_metrics), thresholds, and a short rendering note. Provide formulas for error budget calculation.

Role: You are a Splunk security analyst. Task: provide a structured triage checklist and a set of SPL queries to investigate suspicious IP activity. Constraints: accept input variable 'ip_address', search across index=firewall OR index=proxy OR index=endpoint, lookback 7 days, return timelines, user/host associations, and outbound connections. Output format: numbered triage steps, then 4 SPL queries labeled (summary, timeline, user-host pivots, enrichment), and a short recommended severity and next action (containment, monitor, block). Include one example enrichment command (WHOIS or threat intel lookup) as SPL or pseudo-SPL.

Role: You are a senior SOC engineer designing a Splunk SOAR playbook. Task: produce a multi-step incident response playbook for 'suspicious privilege escalation' that integrates Splunk Enterprise Security, threat intel, and SOAR actions. Constraints: include input triggers (correlation search), decision gates (confidence thresholds), automated enrichments (WHOIS, enrich IOC, asset criticality lookup), containment steps (disable account, isolate host), manual review steps, and post-incident reporting. Output format: ordered JSON with steps: id, name, type (automated/manual), preconditions, action (SPL or SOAR API), success criteria, rollback. Provide two short examples of action payloads.

Role: You are a Splunk platform engineer advising on retention and cost. Task: produce a multi-step index retention and tiering plan based on ingestion rates, compliance windows, and cost targets. Constraints: accept variables average_daily_ingest_GB, retention_days_required, hot_warm_cold_layers boolean, and target_monthly_cost_Budget; compute required storage (with 1.2x compression factor), recommend retention per index, cold/archive options, and expected monthly storage cost estimates. Output format: numbered plan steps, table-like JSON with index name, ingest_GB/day, retention_days, projected_storage_GB, tier, and monthly_cost_estimate, plus brief deployment checklist for index.conf changes.

Compare Splunk with Elastic, Datadog, New Relic. Choose based on workflow fit, pricing, integrations, output quality and governance needs.