Phrase AI translation

A Python tool to interact with Phrase TMS with AI-powered translation capabilities.

Features

Authentication with Phrase TMS
Download bilingual MXLIFF files for all jobs in new projects sent by Argo
Handles pagination automatically for large job lists
Organizes downloaded files by filename and job ID
AI-powered translation using Google's Vertex AI Gemini model
Automatic upload of translated files back to Phrase TMS
Smart handling of XLIFF/MXLIFF tag conversions
Glossary support with term detection using Phrase termbase matching
Parallel processing for improved performance with large files

Setup

Install dependencies:

pip install -r requirements.txt

Configure environment variables in
```
.env
```
:

```
PHRASETMS_USERNAME
```
- Phrase username (not email address)
```
PHRASETMS_PASSWORD
```
- Phrase password
```
PHRASETMS_USE_US_DATA_CENTER
```
- Set to
```
true
```
to use the US data center. Set to
```
false
```
, leave empty or remove altogether to use the EU data center.
```
PHRASETMS_TERM_BASE_UID
```
- Term base UID of the term base in Phrase
```
GOOGLE_SERVICE_ACCOUNT_EMAIL
```
- Google Cloud service account email
```
GOOGLE_SERVICE_ACCOUNT_PRIVATE_KEY
```
- Google Cloud service account private key
```
GOOGLE_CLOUD_PROJECT_ID
```
- Google Cloud project ID
```
GOOGLE_CLOUD_LOCATION_ID
```
- Google Cloud location (e.g.
```
us-central1
```
)
```
GOOGLE_DRIVE_JOBS_UPLOAD_FOLDER_ID
```
- Google Drive folder ID into which project folders with job folders will be uploaded. This needs to be a folder that the service account has access to in a shared drive.

Configuration files:

```
prompt.txt
```
- Create this file in the same directory as script.py with your translation prompt
```
style-guide.md
```
- Optional - Create this file to include style guidelines that will be appended to the translation prompt

The script will automatically detect and incorporate the style guide if present, enhancing translation quality and consistency. If it is not found, the script will continue running with just the base prompt.

Usage

Run the script:

python script.py

Output Structure

```
jobs/
```
- Directory containing all job files
- ```
job_id/
```
  - Individual job directory
  - ```
  input.mxliff
```
  - Original downloaded file
- ```
input_minified.mxliff
```
    - Minified version of the original file that is sent for translation
  - ```
  output.mxliff
```
  - Translated file

Translation Process

Logs into Google Cloud project and initializes the LLM (currently uses Gemini 2.0 Flash)
Logs into Phrase and downloads MXLIFF from all jobs in the specified project
- Filters for projects for "Docs" client with "NEW" status and owned by the "gitlab" (Spartan Software) user
Goes through each downloaded MXLIFF, minifies it (leaves only the group elements that have non-locked trans-units) and replaces all MXLIFF tags with XLIFF inline elements (
```
{1}
```
=>
```
<x id="1"/>
```
and
```
{2>GitLab<2}
```
=>
```
<g id="2">GitLab</g>
```
)
If a glossary is provided, identifies terms used in the source text and includes only those terms in the translation prompt
If a style guide is provided, appends the style guide to the translation prompt
For large files, automatically splits translation into manageable batches (controlled by
```
XLIFF_TRANS_UNIT_BATCH_SIZE
```
constant)
Asks the model to generate the translations for each minified MXLIFF and parses the output
Changes the XLIFF inline elements in translations back to MXLIFF tags, replaces the targets in trans-units and marks each trans-unit as confirmed
Uploads the translated MXLIFF back to Phrase

Testing

To run the tests for this project:

Install test dependencies:

pip install -r requirements.txt
pip install -r test-requirements.txt

Run the tests using
```
pytest
```
:

python -m pytest test_script.py -v

Style Guide

Formatting

When working with the

style-guide.md

file, do not use Prettier for formatting as it adds unwanted spaces between Japanese and English characters. Instead, use

markdownlint

for proper formatting of Markdown files containing Japanese text.

If you're using Visual Studio Code, you'll be prompted to install the recommended markdownlint extension when opening this repository. This extension is configured as the default formatter for Markdown files in the project settings (

.vscode/settings.json

The markdownlint extension preserves the correct spacing conventions for Japanese text, ensuring that no spaces are added between Japanese and English characters, which is critical for maintaining proper Japanese typography as outlined in the style guide.

PDF representation

The

style-guide.md

Markdown file is converted to PDF format to be shared with linguists. This helps them understand the style guide in a more readable format.

This conversion is done using the

md-to-pdf

npm package using the following command:

md-to-pdf style-guide.md --document-title="GitLab Japanese Tech Docs Style Guide"

Phrase AI translation

A Python tool to interact with Phrase TMS with AI-powered translation capabilities.

Features

Authentication with Phrase TMS
Download bilingual MXLIFF files for all jobs in new projects sent by Argo
Handles pagination automatically for large job lists
Organizes downloaded files by filename and job ID
AI-powered translation using Google's Vertex AI Gemini model
Automatic upload of translated files back to Phrase TMS
Smart handling of XLIFF/MXLIFF tag conversions
Glossary support with term detection using Phrase termbase matching
Parallel processing for improved performance with large files

Setup

Install dependencies:

pip install -r requirements.txt

Configure environment variables in
```
.env
```
:

```
PHRASETMS_USERNAME
```
- Phrase username (not email address)
```
PHRASETMS_PASSWORD
```
- Phrase password
```
PHRASETMS_USE_US_DATA_CENTER
```
- Set to
```
true
```
to use the US data center. Set to
```
false
```
, leave empty or remove altogether to use the EU data center.
```
PHRASETMS_TERM_BASE_UID
```
- Term base UID of the term base in Phrase
```
GOOGLE_SERVICE_ACCOUNT_EMAIL
```
- Google Cloud service account email
```
GOOGLE_SERVICE_ACCOUNT_PRIVATE_KEY
```
- Google Cloud service account private key
```
GOOGLE_CLOUD_PROJECT_ID
```
- Google Cloud project ID
```
GOOGLE_CLOUD_LOCATION_ID
```
- Google Cloud location (e.g.
```
us-central1
```
)
```
GOOGLE_DRIVE_JOBS_UPLOAD_FOLDER_ID
```
- Google Drive folder ID into which project folders with job folders will be uploaded. This needs to be a folder that the service account has access to in a shared drive.

Configuration files:

```
prompt.txt
```
- Create this file in the same directory as script.py with your translation prompt
```
style-guide.md
```
- Optional - Create this file to include style guidelines that will be appended to the translation prompt

Usage

Run the script:

python script.py

Output Structure

```
jobs/
```
- Directory containing all job files
- ```
job_id/
```
  - Individual job directory
  - ```
  input.mxliff
```
  - Original downloaded file
- ```
input_minified.mxliff
```
    - Minified version of the original file that is sent for translation
  - ```
  output.mxliff
```
  - Translated file

Translation Process

Logs into Google Cloud project and initializes the LLM (currently uses Gemini 2.0 Flash)
Logs into Phrase and downloads MXLIFF from all jobs in the specified project
- Filters for projects for "Docs" client with "NEW" status and owned by the "gitlab" (Spartan Software) user
Goes through each downloaded MXLIFF, minifies it (leaves only the group elements that have non-locked trans-units) and replaces all MXLIFF tags with XLIFF inline elements (
```
{1}
```
=>
```
<x id="1"/>
```
and
```
{2>GitLab<2}
```
=>
```
<g id="2">GitLab</g>
```
)
If a glossary is provided, identifies terms used in the source text and includes only those terms in the translation prompt
If a style guide is provided, appends the style guide to the translation prompt
For large files, automatically splits translation into manageable batches (controlled by
```
XLIFF_TRANS_UNIT_BATCH_SIZE
```
constant)
Asks the model to generate the translations for each minified MXLIFF and parses the output
Changes the XLIFF inline elements in translations back to MXLIFF tags, replaces the targets in trans-units and marks each trans-unit as confirmed
Uploads the translated MXLIFF back to Phrase

Testing

To run the tests for this project:

Install test dependencies:

pip install -r requirements.txt
pip install -r test-requirements.txt

Run the tests using
```
pytest
```
:

python -m pytest test_script.py -v

Style Guide

Formatting

When working with the

style-guide.md

file, do not use Prettier for formatting as it adds unwanted spaces between Japanese and English characters. Instead, use

markdownlint

for proper formatting of Markdown files containing Japanese text.

.vscode/settings.json

PDF representation

The

style-guide.md

Markdown file is converted to PDF format to be shared with linguists. This helps them understand the style guide in a more readable format.

This conversion is done using the

md-to-pdf

npm package using the following command:

md-to-pdf style-guide.md --document-title="GitLab Japanese Tech Docs Style Guide"

Phrase AI translation

Phrase AI translation

Features

Setup

Usage

Output Structure

Translation Process

Testing

Style Guide

Formatting

PDF representation

Related Skills

<h1 align="center">

2. Apply Deepthink Protocol (reason about dependencies

- Identify gaps

Phrase AI translation

Features

Setup

Usage

Output Structure

Translation Process

Testing

Style Guide

Formatting

PDF representation