mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 01:38:56 -04:00
Adding endpoint to upload android map files (#161252)
## Summary We need a way to decode Android crash's stacktraces so that they can provide meaningful insights to our customers, this is because, due to security reasons, android apps tend to obfuscate their code before publishing it online, making crash reports contain obfuscated names, which don't make any sense before mapping them to the actual source code names. In order to help our customers deobfuscate their stacktraces, we need to allow them to provide us with an R8 map file, which is generated by the code obfuscation tool (R8) at compile time. This map file is needed to later do the deobfuscation process. So these code changes take care of adding a new endpoint that our customers can use to upload their map files, similarly to what's currently available to [RUM Sourcemaps](https://www.elastic.co/guide/en/apm/guide/current/source-map-how-to.html#source-map-rum-upload), the Android map files will be uploaded to ES, using the same index as the one currently used to store RUM Sourcemaps. There's a couple of reasons why a new endpoint to upload android maps is needed instead of re-using the existing RUM Sourcemaps one: * The Sourcemaps upload endpoint has validations in place to check the sourcemap format, which must be a JSON with some expected keys available. Android map files don't have a JSON format, so they are rejected by the sourcemaps endpoint. * Android map files tend to be large in size, just as an example, the map file generated for our [sample app](https://github.com/elastic/opbeans-android) has a size of ~7 MB, so for real apps this number can be larger, which would also cause issues with the RUM upload endpoint since it has a max file limit size of 1 MB. * The RUM upload endpoint contains a parameter (`bundle_filepath `) that doesn't have an equivalent for the android map use case. This PR depends on https://github.com/elastic/kibana/pull/161152 ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com> Co-authored-by: Brandon Morelli <bmorelli25@gmail.com> Co-authored-by: Katerina Patticha <kate@kpatticha.com>
This commit is contained in:
parent
851f40c9fe
commit
6751324a80
7 changed files with 565 additions and 9 deletions
|
@ -11,6 +11,7 @@ Some APM app features are provided via a REST API:
|
|||
* <<agent-config-api>>
|
||||
* <<apm-annotation-api>>
|
||||
* <<rum-sourcemap-api>>
|
||||
* <<android-sourcemap-api>>
|
||||
* <<agent-key-api>>
|
||||
|
||||
[float]
|
||||
|
@ -715,6 +716,219 @@ curl -X DELETE "http://localhost:5601/api/apm/sourcemaps/apm:foo-1.0.0-644fd5a9"
|
|||
*******************************************************
|
||||
////
|
||||
|
||||
[role="xpack"]
|
||||
[[android-sourcemap-api]]
|
||||
=== Android source map API
|
||||
|
||||
IMPORTANT: This endpoint is only compatible with the
|
||||
{apm-guide-ref}/index.html[APM integration for Elastic Agent].
|
||||
|
||||
An Android source map (generated using Android's https://developer.android.com/build/shrink-code[R8 tool])
|
||||
allows obfuscated app stacktraces to be mapped back to original source code --
|
||||
allowing you to maintain the size and security of minimized code, without losing the ability to debug your application.
|
||||
|
||||
For best results, uploading source maps should become a part of your deployment procedure,
|
||||
and not something you only do when you see unhelpful errors.
|
||||
That’s because uploading source maps after errors happen won’t make old errors magically readable --
|
||||
errors must occur again for source mapping to occur.
|
||||
|
||||
The following APIs are available:
|
||||
|
||||
* <<android-sourcemap-post>>
|
||||
* <<android-sourcemap-get>>
|
||||
* <<android-sourcemap-delete>>
|
||||
|
||||
[float]
|
||||
[[use-android-sourcemap-api]]
|
||||
==== How to use APM APIs
|
||||
|
||||
.Expand for required headers, privileges, and usage details
|
||||
[%collapsible%closed]
|
||||
======
|
||||
include::api.asciidoc[tag=using-the-APIs]
|
||||
======
|
||||
|
||||
////
|
||||
*******************************************************
|
||||
////
|
||||
|
||||
[[android-sourcemap-post]]
|
||||
==== Create or update an Android source map
|
||||
|
||||
Create or update an Android source map for a specific app and version.
|
||||
|
||||
[[android-sourcemap-post-privs]]
|
||||
===== Privileges
|
||||
|
||||
The user accessing this endpoint requires `All` Kibana privileges for the {beat_kib_app} feature.
|
||||
For more information, see <<kibana-privileges>>.
|
||||
|
||||
[[android-sourcemap-post-req]]
|
||||
===== Request
|
||||
|
||||
`POST /api/apm/androidmaps`
|
||||
|
||||
[role="child_attributes"]
|
||||
[[android-sourcemap-post-req-body]]
|
||||
===== Request body
|
||||
|
||||
`service_name`::
|
||||
(required, string) The name of the Android app that the map should apply to.
|
||||
|
||||
`service_version`::
|
||||
(required, string) The version of the Android app that the map should apply to.
|
||||
|
||||
`map_file`::
|
||||
(required, string or file upload) The R8-generated map.
|
||||
|
||||
[[android-sourcemap-post-example]]
|
||||
===== Examples
|
||||
|
||||
The following example uploads a source map for a app named `foo` and a service version of `1.0.0`:
|
||||
|
||||
[source,curl]
|
||||
--------------------------------------------------
|
||||
curl -X POST "http://localhost:5601/api/apm/androidmaps" \
|
||||
-H 'Content-Type: multipart/form-data' \
|
||||
-H 'kbn-xsrf: true' \
|
||||
-H 'Authorization: ApiKey ${YOUR_API_KEY}' \
|
||||
-F 'service_name="foo"' \
|
||||
-F 'service_version="1.0.0"' \
|
||||
-F 'map_file=@"/Path/to/the/file/mapping.txt"'
|
||||
--------------------------------------------------
|
||||
|
||||
[[android-sourcemap-post-body]]
|
||||
===== Response body
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"type": "sourcemap",
|
||||
"identifier": "foo-1.0.0-android",
|
||||
"relative_url": "/api/fleet/artifacts/foo-1.0.0-android/644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"body": "eJyFkL1OwzAUhd/Fc+MbYMuCEBIbHRjKgBgc96R16tiWr1OQqr47NwqJxEK3q/PzWccXxchnZ7E1A1SjuhjVZtF2yOxiEPlO17oWox3D3uPFeSRTjmJQARfCPeiAgGx8NTKsYdAc1T3rwaSJGcds8Sp3c1HnhfywUZ3QhMTFFGepZxqMC9oex3CS9tpk1XyozgOlmoVKuJX1DqEQZ0su7PGtLU+V/3JPKc3cL7TJ2FNDRPov4bFta3MDM4f7W69lpJjLO9qdK8bzVPhcJz3HUCQ4LbO/p5hCSC4cZPByrp/wFqOklbpefwAhzpqI",
|
||||
"created": "2021-07-09T20:47:44.812Z",
|
||||
"id": "apm:foo-1.0.0-android-644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"compressionAlgorithm": "zlib",
|
||||
"decodedSha256": "644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"decodedSize": 441,
|
||||
"encodedSha256": "024c72749c3e3dd411b103f7040ae62633558608f480bce4b108cf5b2275bd24",
|
||||
"encodedSize": 237,
|
||||
"encryptionAlgorithm": "none",
|
||||
"packageName": "apm"
|
||||
}
|
||||
--------------------------------------------------
|
||||
|
||||
////
|
||||
*******************************************************
|
||||
////
|
||||
|
||||
[[android-sourcemap-get]]
|
||||
==== Get source maps
|
||||
|
||||
Returns an array of Fleet artifacts, including source map uploads.
|
||||
|
||||
[[android-sourcemap-get-privs]]
|
||||
===== Privileges
|
||||
|
||||
The user accessing this endpoint requires `Read` or `All` Kibana privileges for the {beat_kib_app} feature.
|
||||
For more information, see <<kibana-privileges>>.
|
||||
|
||||
[[android-sourcemap-get-req]]
|
||||
===== Request
|
||||
|
||||
`GET /api/apm/sourcemaps`
|
||||
|
||||
[[android-sourcemap-get-example]]
|
||||
===== Example
|
||||
|
||||
The following example requests all uploaded source maps:
|
||||
|
||||
[source,curl]
|
||||
--------------------------------------------------
|
||||
curl -X GET "http://localhost:5601/api/apm/sourcemaps" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'kbn-xsrf: true' \
|
||||
-H 'Authorization: ApiKey ${YOUR_API_KEY}'
|
||||
--------------------------------------------------
|
||||
|
||||
[[android-sourcemap-get-body]]
|
||||
===== Response body
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"artifacts": [
|
||||
{
|
||||
"type": "sourcemap",
|
||||
"identifier": "foo-1.0.0-android",
|
||||
"relative_url": "/api/fleet/artifacts/foo-1.0.0-android/644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"body": {
|
||||
"serviceName": "foo",
|
||||
"serviceVersion": "1.0.0",
|
||||
"bundleFilepath": "android",
|
||||
"sourceMap": "# compiler: R8\n# compiler_version: 3.2.47\n# min_api: 26\n..."
|
||||
},
|
||||
"created": "2021-07-09T20:47:44.812Z",
|
||||
"id": "apm:foo-1.0.0-android-644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"compressionAlgorithm": "zlib",
|
||||
"decodedSha256": "644fd5a997d1ddd90ee131ba18e2b3d03931d89dd1fe4599143c0b3264b3e456",
|
||||
"decodedSize": 441,
|
||||
"encodedSha256": "024c72749c3e3dd411b103f7040ae62633558608f480bce4b108cf5b2275bd24",
|
||||
"encodedSize": 237,
|
||||
"encryptionAlgorithm": "none",
|
||||
"packageName": "apm"
|
||||
}
|
||||
]
|
||||
}
|
||||
--------------------------------------------------
|
||||
|
||||
////
|
||||
*******************************************************
|
||||
////
|
||||
|
||||
[[android-sourcemap-delete]]
|
||||
==== Delete source map
|
||||
|
||||
Delete a previously uploaded source map.
|
||||
|
||||
[[android-sourcemap-delete-privs]]
|
||||
===== Privileges
|
||||
|
||||
The user accessing this endpoint requires `All` Kibana privileges for the {beat_kib_app} feature.
|
||||
For more information, see <<kibana-privileges>>.
|
||||
|
||||
[[android-sourcemap-delete-req]]
|
||||
===== Request
|
||||
|
||||
`DELETE /api/apm/sourcemaps/:id`
|
||||
|
||||
[[android-sourcemap-delete-example]]
|
||||
===== Example
|
||||
|
||||
The following example deletes a source map with an id of `apm:foo-1.0.0-android-644fd5a9`:
|
||||
|
||||
[source,curl]
|
||||
--------------------------------------------------
|
||||
curl -X DELETE "http://localhost:5601/api/apm/sourcemaps/apm:foo-1.0.0-android-644fd5a9" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'kbn-xsrf: true' \
|
||||
-H 'Authorization: ApiKey ${YOUR_API_KEY}'
|
||||
--------------------------------------------------
|
||||
|
||||
[[android-sourcemap-delete-body]]
|
||||
===== Response body
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{}
|
||||
--------------------------------------------------
|
||||
|
||||
////
|
||||
*******************************************************
|
||||
*******************************************************
|
||||
////
|
||||
|
||||
[role="xpack"]
|
||||
[[agent-key-api]]
|
||||
=== APM agent Key API
|
||||
|
|
|
@ -19,12 +19,18 @@ import { getPackagePolicyWithSourceMap } from './get_package_policy_decorators';
|
|||
|
||||
const doUnzip = promisify(unzip);
|
||||
|
||||
interface ApmSourceMapArtifactBody {
|
||||
interface ApmMapArtifactBody {
|
||||
serviceName: string;
|
||||
serviceVersion: string;
|
||||
bundleFilepath: string;
|
||||
sourceMap: string;
|
||||
}
|
||||
|
||||
interface ApmSourceMapArtifactBody
|
||||
extends Omit<ApmMapArtifactBody, 'sourceMap'> {
|
||||
sourceMap: SourceMap;
|
||||
}
|
||||
|
||||
export type ArtifactSourceMap = Omit<Artifact, 'body'> & {
|
||||
body: ApmSourceMapArtifactBody;
|
||||
};
|
||||
|
@ -104,6 +110,23 @@ export async function createFleetSourceMapArtifact({
|
|||
});
|
||||
}
|
||||
|
||||
export async function createFleetAndroidMapArtifact({
|
||||
apmArtifactBody,
|
||||
fleetPluginStart,
|
||||
}: {
|
||||
apmArtifactBody: ApmMapArtifactBody;
|
||||
fleetPluginStart: FleetPluginStart;
|
||||
}) {
|
||||
const apmArtifactClient = getApmArtifactClient(fleetPluginStart);
|
||||
const identifier = `${apmArtifactBody.serviceName}-${apmArtifactBody.serviceVersion}-android`;
|
||||
|
||||
return apmArtifactClient.createArtifact({
|
||||
type: 'sourcemap',
|
||||
identifier,
|
||||
content: JSON.stringify(apmArtifactBody),
|
||||
});
|
||||
}
|
||||
|
||||
export async function deleteFleetSourcemapArtifact({
|
||||
id,
|
||||
fleetPluginStart,
|
||||
|
|
|
@ -10,7 +10,7 @@ import { Artifact } from '@kbn/fleet-plugin/server';
|
|||
import { getUnzippedArtifactBody } from '../fleet/source_maps';
|
||||
import { APM_SOURCE_MAP_INDEX } from '../settings/apm_indices/get_apm_indices';
|
||||
import { ApmSourceMap } from './create_apm_source_map_index_template';
|
||||
import { getEncodedContent, getSourceMapId } from './sourcemap_utils';
|
||||
import { getEncodedSourceMapContent, getSourceMapId } from './sourcemap_utils';
|
||||
|
||||
export async function bulkCreateApmSourceMaps({
|
||||
artifacts,
|
||||
|
@ -24,7 +24,7 @@ export async function bulkCreateApmSourceMaps({
|
|||
const { serviceName, serviceVersion, bundleFilepath, sourceMap } =
|
||||
await getUnzippedArtifactBody(artifact.body);
|
||||
|
||||
const { contentEncoded, contentHash } = await getEncodedContent(
|
||||
const { contentEncoded, contentHash } = await getEncodedSourceMapContent(
|
||||
sourceMap
|
||||
);
|
||||
|
||||
|
|
|
@ -10,7 +10,11 @@ import { Logger } from '@kbn/core/server';
|
|||
import { APM_SOURCE_MAP_INDEX } from '../settings/apm_indices/get_apm_indices';
|
||||
import { ApmSourceMap } from './create_apm_source_map_index_template';
|
||||
import { SourceMap } from './route';
|
||||
import { getEncodedContent, getSourceMapId } from './sourcemap_utils';
|
||||
import {
|
||||
getEncodedSourceMapContent,
|
||||
getEncodedContent,
|
||||
getSourceMapId,
|
||||
} from './sourcemap_utils';
|
||||
|
||||
export async function createApmSourceMap({
|
||||
internalESClient,
|
||||
|
@ -31,9 +35,75 @@ export async function createApmSourceMap({
|
|||
serviceName: string;
|
||||
serviceVersion: string;
|
||||
}) {
|
||||
const { contentEncoded, contentHash } = await getEncodedContent(
|
||||
const { contentEncoded, contentHash } = await getEncodedSourceMapContent(
|
||||
sourceMapContent
|
||||
);
|
||||
return await doCreateApmMap({
|
||||
internalESClient,
|
||||
logger,
|
||||
fleetId,
|
||||
created,
|
||||
bundleFilepath,
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
contentEncoded,
|
||||
contentHash,
|
||||
});
|
||||
}
|
||||
|
||||
export async function createApmAndroidMap({
|
||||
internalESClient,
|
||||
logger,
|
||||
fleetId,
|
||||
created,
|
||||
mapContent,
|
||||
bundleFilepath,
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
}: {
|
||||
internalESClient: ElasticsearchClient;
|
||||
logger: Logger;
|
||||
fleetId: string;
|
||||
created: string;
|
||||
mapContent: string;
|
||||
bundleFilepath: string;
|
||||
serviceName: string;
|
||||
serviceVersion: string;
|
||||
}) {
|
||||
const { contentEncoded, contentHash } = await getEncodedContent(mapContent);
|
||||
return await doCreateApmMap({
|
||||
internalESClient,
|
||||
logger,
|
||||
fleetId,
|
||||
created,
|
||||
bundleFilepath,
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
contentEncoded,
|
||||
contentHash,
|
||||
});
|
||||
}
|
||||
async function doCreateApmMap({
|
||||
internalESClient,
|
||||
logger,
|
||||
fleetId,
|
||||
created,
|
||||
bundleFilepath,
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
contentEncoded,
|
||||
contentHash,
|
||||
}: {
|
||||
internalESClient: ElasticsearchClient;
|
||||
logger: Logger;
|
||||
fleetId: string;
|
||||
created: string;
|
||||
bundleFilepath: string;
|
||||
serviceName: string;
|
||||
serviceVersion: string;
|
||||
contentEncoded: string;
|
||||
contentHash: string;
|
||||
}) {
|
||||
const doc: ApmSourceMap = {
|
||||
fleet_id: fleetId,
|
||||
created,
|
||||
|
|
|
@ -9,19 +9,24 @@ import { SavedObjectsClientContract } from '@kbn/core/server';
|
|||
import { Artifact } from '@kbn/fleet-plugin/server';
|
||||
import { jsonRt, toNumberRt } from '@kbn/io-ts-utils';
|
||||
import * as t from 'io-ts';
|
||||
import { either } from 'fp-ts/lib/Either';
|
||||
import { ApmFeatureFlags } from '../../../common/apm_feature_flags';
|
||||
import { getInternalSavedObjectsClient } from '../../lib/helpers/get_internal_saved_objects_client';
|
||||
import { stringFromBufferRt } from '../../utils/string_from_buffer_rt';
|
||||
import { createApmServerRoute } from '../apm_routes/create_apm_server_route';
|
||||
import {
|
||||
createFleetSourceMapArtifact,
|
||||
createFleetAndroidMapArtifact,
|
||||
deleteFleetSourcemapArtifact,
|
||||
getCleanedBundleFilePath,
|
||||
listSourceMapArtifacts,
|
||||
ListSourceMapArtifactsResponse,
|
||||
updateSourceMapsOnFleetPolicies,
|
||||
} from '../fleet/source_maps';
|
||||
import { createApmSourceMap } from './create_apm_source_map';
|
||||
import {
|
||||
createApmSourceMap,
|
||||
createApmAndroidMap,
|
||||
} from './create_apm_source_map';
|
||||
import { deleteApmSourceMap } from './delete_apm_sourcemap';
|
||||
import { runFleetSourcemapArtifactsMigration } from './schedule_source_map_migration';
|
||||
|
||||
|
@ -41,6 +46,24 @@ export const sourceMapRt = t.intersection([
|
|||
|
||||
export type SourceMap = t.TypeOf<typeof sourceMapRt>;
|
||||
|
||||
const androidMapValidation = new t.Type<string, string, unknown>(
|
||||
'ANDROID_MAP_VALIDATION',
|
||||
t.string.is,
|
||||
(input, context): t.Validation<string> =>
|
||||
either.chain(
|
||||
t.string.validate(input, context),
|
||||
(str): t.Validation<string> => {
|
||||
const firstLine = str.split('\n', 1)[0];
|
||||
if (firstLine.trim() === '# compiler: R8') {
|
||||
return t.success(str);
|
||||
} else {
|
||||
return t.failure(input, context);
|
||||
}
|
||||
}
|
||||
),
|
||||
(a): string => a
|
||||
);
|
||||
|
||||
function throwNotImplementedIfSourceMapNotAvailable(
|
||||
featureFlags: ApmFeatureFlags
|
||||
): void {
|
||||
|
@ -91,7 +114,7 @@ const uploadSourceMapRoute = createApmServerRoute({
|
|||
endpoint: 'POST /api/apm/sourcemaps 2023-10-31',
|
||||
options: {
|
||||
tags: ['access:apm', 'access:apm_write'],
|
||||
body: { accepts: ['multipart/form-data'] },
|
||||
body: { accepts: ['multipart/form-data'], maxBytes: 100 * 1024 * 1024 },
|
||||
},
|
||||
params: t.type({
|
||||
body: t.type({
|
||||
|
@ -169,6 +192,85 @@ const uploadSourceMapRoute = createApmServerRoute({
|
|||
},
|
||||
});
|
||||
|
||||
const uploadAndroidMapRoute = createApmServerRoute({
|
||||
endpoint: 'POST /api/apm/androidmaps 2023-10-31',
|
||||
options: {
|
||||
tags: ['access:apm', 'access:apm_write'],
|
||||
body: { accepts: ['multipart/form-data'], maxBytes: 100 * 1024 * 1024 },
|
||||
},
|
||||
params: t.type({
|
||||
body: t.type({
|
||||
service_name: t.string,
|
||||
service_version: t.string,
|
||||
map_file: t
|
||||
.union([t.string, stringFromBufferRt])
|
||||
.pipe(androidMapValidation),
|
||||
}),
|
||||
}),
|
||||
handler: async ({
|
||||
params,
|
||||
plugins,
|
||||
core,
|
||||
logger,
|
||||
featureFlags,
|
||||
}): Promise<Artifact | undefined> => {
|
||||
throwNotImplementedIfSourceMapNotAvailable(featureFlags);
|
||||
|
||||
const {
|
||||
service_name: serviceName,
|
||||
service_version: serviceVersion,
|
||||
map_file: sourceMapContent,
|
||||
} = params.body;
|
||||
const bundleFilepath = 'android';
|
||||
const fleetPluginStart = await plugins.fleet?.start();
|
||||
const coreStart = await core.start();
|
||||
const internalESClient = coreStart.elasticsearch.client.asInternalUser;
|
||||
const savedObjectsClient = await getInternalSavedObjectsClient(coreStart);
|
||||
try {
|
||||
if (fleetPluginStart) {
|
||||
// create source map as fleet artifact
|
||||
const artifact = await createFleetAndroidMapArtifact({
|
||||
fleetPluginStart,
|
||||
apmArtifactBody: {
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
bundleFilepath,
|
||||
sourceMap: sourceMapContent,
|
||||
},
|
||||
});
|
||||
|
||||
// sync source map to APM managed index
|
||||
await createApmAndroidMap({
|
||||
internalESClient,
|
||||
logger,
|
||||
fleetId: artifact.id,
|
||||
created: artifact.created,
|
||||
mapContent: sourceMapContent,
|
||||
bundleFilepath,
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
});
|
||||
|
||||
// sync source map to fleet policy
|
||||
await updateSourceMapsOnFleetPolicies({
|
||||
coreStart,
|
||||
fleetPluginStart,
|
||||
savedObjectsClient:
|
||||
savedObjectsClient as unknown as SavedObjectsClientContract,
|
||||
internalESClient,
|
||||
});
|
||||
|
||||
return artifact;
|
||||
}
|
||||
} catch (e) {
|
||||
throw Boom.internal(
|
||||
'Something went wrong while creating a new android map',
|
||||
e
|
||||
);
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
const deleteSourceMapRoute = createApmServerRoute({
|
||||
endpoint: 'DELETE /api/apm/sourcemaps/{id} 2023-10-31',
|
||||
options: { tags: ['access:apm', 'access:apm_write'] },
|
||||
|
@ -230,5 +332,6 @@ export const sourceMapsRouteRepository = {
|
|||
...listSourceMapRoute,
|
||||
...uploadSourceMapRoute,
|
||||
...deleteSourceMapRoute,
|
||||
...uploadAndroidMapRoute,
|
||||
...migrateFleetArtifactsSourceMapRoute,
|
||||
};
|
||||
|
|
|
@ -16,8 +16,11 @@ function asSha256Encoded(content: BinaryLike): string {
|
|||
return createHash('sha256').update(content).digest('hex');
|
||||
}
|
||||
|
||||
export async function getEncodedContent(sourceMapContent: SourceMap) {
|
||||
const contentBuffer = Buffer.from(JSON.stringify(sourceMapContent));
|
||||
export async function getEncodedSourceMapContent(sourceMapContent: SourceMap) {
|
||||
return getEncodedContent(JSON.stringify(sourceMapContent));
|
||||
}
|
||||
export async function getEncodedContent(textContent: string) {
|
||||
const contentBuffer = Buffer.from(textContent);
|
||||
const contentZipped = await deflateAsync(contentBuffer);
|
||||
const contentEncoded = contentZipped.toString('base64');
|
||||
const contentHash = asSha256Encoded(contentZipped);
|
||||
|
|
|
@ -27,6 +27,16 @@ const SAMPLE_SOURCEMAP = {
|
|||
mappings: 'A,AAAB;;ABCDE;',
|
||||
};
|
||||
|
||||
const SAMPLE_ANDROID_MAP = `# compiler: R8
|
||||
# compiler_version: 3.2.47
|
||||
# min_api: 26
|
||||
# common_typos_disable
|
||||
# {"id":"com.android.tools.r8.mapping","version":"2.0"}
|
||||
# pg_map_id: 127b14c
|
||||
# pg_map_hash: SHA-256 127b14c0be5dd1b55beee544a8d0e7c9414b432868ed8bc54ca5cc43cba12435
|
||||
a1.TableInfo$ForeignKey$$ExternalSyntheticOutline0 -> a1.e:
|
||||
# {"id":"sourceFile","fileName":"R8$$SyntheticClass"}`;
|
||||
|
||||
export default function ApiTest({ getService }: FtrProviderContext) {
|
||||
const registry = getService('registry');
|
||||
const apmApiClient = getService('apmApiClient');
|
||||
|
@ -99,6 +109,28 @@ export default function ApiTest({ getService }: FtrProviderContext) {
|
|||
return response.body;
|
||||
}
|
||||
|
||||
async function uploadAndroidMap({
|
||||
serviceName,
|
||||
serviceVersion,
|
||||
androidMap,
|
||||
}: {
|
||||
serviceName: string;
|
||||
serviceVersion: string;
|
||||
androidMap: string;
|
||||
}) {
|
||||
const response = await apmApiClient.writeUser({
|
||||
endpoint: 'POST /api/apm/androidmaps 2023-10-31',
|
||||
type: 'form-data',
|
||||
params: {
|
||||
body: {
|
||||
service_name: serviceName,
|
||||
service_version: serviceVersion,
|
||||
map_file: androidMap,
|
||||
},
|
||||
},
|
||||
});
|
||||
return response.body;
|
||||
}
|
||||
async function runSourceMapMigration() {
|
||||
await apmApiClient.writeUser({
|
||||
endpoint: 'POST /internal/apm/sourcemaps/migrate_fleet_artifacts',
|
||||
|
@ -128,6 +160,12 @@ export default function ApiTest({ getService }: FtrProviderContext) {
|
|||
await Promise.all([deleteAllFleetSourceMaps(), deleteAllApmSourceMaps()]);
|
||||
});
|
||||
|
||||
async function getDecodedMapContent(encodedContent?: string): Promise<string | undefined> {
|
||||
if (encodedContent) {
|
||||
return (await unzip(Buffer.from(encodedContent, 'base64'))).toString();
|
||||
}
|
||||
}
|
||||
|
||||
async function getDecodedSourceMapContent(
|
||||
encodedContent?: string
|
||||
): Promise<SourceMap | undefined> {
|
||||
|
@ -243,6 +281,111 @@ export default function ApiTest({ getService }: FtrProviderContext) {
|
|||
});
|
||||
});
|
||||
|
||||
let androidResp: APIReturnType<'POST /api/apm/androidmaps 2023-10-31'>;
|
||||
describe('upload android map', () => {
|
||||
after(async () => {
|
||||
await apmApiClient.writeUser({
|
||||
endpoint: 'DELETE /api/apm/sourcemaps/{id} 2023-10-31',
|
||||
params: { path: { id: androidResp.id } },
|
||||
});
|
||||
});
|
||||
|
||||
before(async () => {
|
||||
androidResp = await uploadAndroidMap({
|
||||
serviceName: 'uploading-test',
|
||||
serviceVersion: '1.0.0',
|
||||
androidMap: SAMPLE_ANDROID_MAP,
|
||||
});
|
||||
|
||||
await waitForSourceMapCount(1);
|
||||
});
|
||||
|
||||
it('is uploaded as a fleet artifact', async () => {
|
||||
const res = await esClient.search({
|
||||
index: '.fleet-artifacts',
|
||||
size: 1,
|
||||
query: {
|
||||
bool: {
|
||||
filter: [{ term: { type: 'sourcemap' } }, { term: { package_name: 'apm' } }],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
// @ts-expect-error
|
||||
expect(res.hits.hits[0]._source.identifier).to.be('uploading-test-1.0.0-android');
|
||||
});
|
||||
|
||||
it('is added to .apm-source-map index', async () => {
|
||||
const res = await esClient.search<ApmSourceMap>({
|
||||
index: '.apm-source-map',
|
||||
});
|
||||
|
||||
const source = res.hits.hits[0]._source;
|
||||
const decodedSourceMap = await getDecodedMapContent(source?.content);
|
||||
expect(decodedSourceMap).to.eql(SAMPLE_ANDROID_MAP);
|
||||
expect(source?.content_sha256).to.be(
|
||||
'702e07279b0fbed47fdbf5e71528dff845b4f07a16ca79cab0c1b06eb71be966'
|
||||
);
|
||||
expect(source?.file.path).to.be('android');
|
||||
expect(source?.service.name).to.be('uploading-test');
|
||||
expect(source?.service.version).to.be('1.0.0');
|
||||
});
|
||||
|
||||
describe('when uploading a new android map with the same service.name and service.version', () => {
|
||||
let resBefore: GetResponse<ApmSourceMap>;
|
||||
let resAfter: GetResponse<ApmSourceMap>;
|
||||
|
||||
before(async () => {
|
||||
async function getSourceMapDocFromApmIndex() {
|
||||
await esClient.indices.refresh({ index: '.apm-source-map' });
|
||||
return await esClient.get<ApmSourceMap>({
|
||||
index: '.apm-source-map',
|
||||
id: 'uploading-test-1.0.0-android',
|
||||
});
|
||||
}
|
||||
|
||||
resBefore = await getSourceMapDocFromApmIndex();
|
||||
|
||||
await uploadAndroidMap({
|
||||
serviceName: 'uploading-test',
|
||||
serviceVersion: '1.0.0',
|
||||
androidMap: '# compiler: R8\n# ANOTHER MAP',
|
||||
});
|
||||
|
||||
resAfter = await getSourceMapDocFromApmIndex();
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
await deleteAllApmSourceMaps();
|
||||
await deleteAllFleetSourceMaps();
|
||||
});
|
||||
|
||||
it('creates one document in the .apm-source-map index', async () => {
|
||||
const res = await esClient.search<ApmSourceMap>({ index: '.apm-source-map', size: 0 });
|
||||
|
||||
// @ts-expect-error
|
||||
expect(res.hits.total.value).to.be(1);
|
||||
});
|
||||
|
||||
it('creates two documents in the .fleet-artifacts index', async () => {
|
||||
const res = await listSourcemaps({ page: 1, perPage: 10 });
|
||||
expect(res.total).to.be(2);
|
||||
});
|
||||
|
||||
it('updates the content', async () => {
|
||||
const contentBefore = await getDecodedMapContent(resBefore._source?.content);
|
||||
const contentAfter = await getDecodedMapContent(resAfter._source?.content);
|
||||
|
||||
expect(contentBefore).to.be(SAMPLE_ANDROID_MAP);
|
||||
expect(contentAfter).to.be('# compiler: R8\n# ANOTHER MAP');
|
||||
});
|
||||
|
||||
it('updates the content hash', async () => {
|
||||
expect(resBefore._source?.content_sha256).to.not.be(resAfter._source?.content_sha256);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('list source maps', async () => {
|
||||
before(async () => {
|
||||
const totalCount = 6;
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue