DeepSeek API: an independent guide and a drop-in alternative

Understanding Token Limits

Token limits define the boundaries of your model's working memory. It is crucial to understand that the context window includes both the input tokens (your prompt and history) and the output tokens (the model's response). If your combined total exceeds the limit, the request will fail.

For example, if a model has a 64,000-token context window and you send a 50,000-token prompt, you only have 14,000 tokens left for the response. Many developers miscalculate this by ignoring the input size, leading to unexpected errors. Always monitor your token usage closely, especially when dealing with long conversation histories or large codebases.

• Input tokens: Your prompt, system instructions, and previous messages.
• Output tokens: The model's generated text.
• Total usage: Input + Output must be ≤ Context Window Limit.

Fixing Streaming Errors

Streaming allows you to receive tokens as they are generated, providing a better user experience for applications. However, streaming can fail due to network issues, client-side parsing errors, or server-side timeouts. When streaming fails, the connection may drop prematurely, leaving your application with incomplete data.

To fix streaming errors, ensure your client correctly handles Server-Sent Events (SSE). Each chunk should be parsed incrementally, and the final chunk often contains the total token usage. If the connection drops, you may need to implement retry logic with exponential backoff. Additionally, verify that your network infrastructure does not interfere with long-lived HTTP connections.

Common causes of streaming errors include:

• Client-side timeouts that are too short.
• Incorrect parsing of SSE format.
• Network interruptions during long generation tasks.

Handling Context Window Exceeded

When the context window is exceeded, the model cannot process the request because it exceeds its maximum capacity. This is a hard limit defined by the model architecture. To handle this, you need to implement strategies to reduce the token count.

One effective strategy is to truncate older messages in the conversation history. Another is to summarize previous interactions and replace them with a condensed version. You can also reduce the verbosity of your system instructions or remove unnecessary details from the prompt. By carefully managing token usage, you can avoid these errors and ensure smoother operation.

Techniques to manage context window limits:

• Truncate old messages.
• Summarize conversation history.
• Optimize prompt structure for brevity.

JSON Mode Validation Issues

JSON mode ensures the model's output is valid JSON, which is essential for programmatic integration. However, validation issues can arise if the model generates trailing text or invalid characters outside the JSON structure. This often happens when the model is not strictly constrained or when the prompt does not clearly define the expected format.

To resolve validation issues, ensure your prompt explicitly requests JSON output and validates the response on the client side. If the model includes trailing text, you may need to parse the JSON manually or use a library that can handle partial JSON structures. Additionally, consider using a stricter prompt to guide the model toward generating clean JSON.

Common JSON validation issues:

• Trailing commas or invalid characters.
• Mismatched data types in the output.
• Extra text outside the JSON object.

Function Calling Failures

Function calling allows the model to invoke predefined functions based on user input. Failures can occur if the function schema is incorrectly defined, the model misinterprets the parameters, or the function execution returns an error. These issues can disrupt the flow of your application and require careful debugging.

To fix function calling failures, ensure your function schema is accurate and comprehensive. Validate the model's response against the schema before executing the function. If the model provides incorrect parameters, you may need to refine the prompt or provide additional examples in the system instructions. Additionally, handle errors gracefully by providing fallback responses or retrying the function call.

Steps to resolve function calling failures:

• Verify function schema accuracy.
• Validate model response against schema.
• Refine prompts for better parameter extraction.

Authentication & Key Rotation

Authentication is the process of verifying the identity of the user or application making the API request. API keys are the primary method of authentication, and they should be kept secure to prevent unauthorized access. Key rotation involves periodically generating new keys and revoking old ones to minimize the risk of compromise.

For our API, each account is associated with one active key. If you generate a new key, the old one is immediately invalidated. This ensures that only the most recent key is used for requests. It is important to update your client applications promptly after key rotation to avoid service disruptions.

Best practices for API key management:

• Store keys securely in environment variables.
• Rotate keys periodically.
• Monitor key usage for anomalies.

Billing & Credit Deductions

Our billing system is transparent and prepaid. You purchase credits, and they are deducted based on actual token usage. Errors and refusals are free, meaning you only pay for successful completions. Credits never expire, providing flexibility in how you manage your budget.

Credits can be topped up using cryptocurrency, specifically USDT (TRC20) or USDC (Base). There are no monthly fees or subscriptions, and you can top up any whole amount between $10 and $500. For larger top-ups, you receive bonus credits: +5% for $50 and +10% for $100. This system ensures you only pay for what you use.

Billing highlights:

• Prepaid credit system.
• No monthly fees or subscriptions.
• Credits never expire.

Rate Limiting Explained

Rate limiting controls the number of requests you can make within a specific time frame. Our API enforces a limit of 300 requests per minute per key and allows up to 8 concurrent requests. These limits prevent abuse and ensure fair usage across all users.

If you exceed the rate limit, your requests may be delayed or rejected. To manage rate limits effectively, implement retry logic with exponential backoff in your client applications. Monitor your usage patterns and adjust your request frequency accordingly. If you need higher limits, consider optimizing your prompts to reduce the number of requests required.

Rate limit details:

• 300 requests per minute per key.
• 8 concurrent requests per key.
• 8 MB request body limit.