Core Scientific Shareholders Reject $9 Billion Acquisition by CoreWeave

Core Scientific shareholders on Thursday voted against a $9 billion all-stock acquisition offer from partner and competitor CoreWeave.

Major Shareholder Influence in Voting Decision

The decision came after a recommendation from the largest shareholder, Sina Toussi of Two Seas Capital, a firm specializing in post-bankruptcy companies. Core Scientific emerged from bankruptcy in January 2024.

Shared Origins: From Crypto Mining to AI

Core Scientific started as a cryptocurrency mining company, a path it also shares with AI data center provider CoreWeave. The latter has evolved to focus on AI workloads, aided by its partnership with investor Nvidia. Since its IPO, CoreWeave’s market cap has skyrocketed from $14 billion to $66 billion (approximately $140 per share), attracting investor interest in AI advancements.

CoreWeave’s Contract and Acquisition Strategies

CoreWeave previously signed a $10 billion, 12-year contract with Core Scientific to leverage its facilities for AI services and had also announced plans to acquire the company outright, offering a premium over Core Scientific’s stock price.

Strategic Outlook from Investors

However, investor Toussi believes Core Scientific has the potential to independently thrive in the AI sector. In his opposition letter, he stated, “Since the transaction was announced in July, investment in AI infrastructure has accelerated, driving equity valuations of Core Scientific’s peers to new heights. Why would anyone vote for a transaction worth only $16.40 per share?”

Market Reaction and Future Prospects

Following the rejection of the deal, Core Scientific’s stock experienced an uptick, reflecting a current market cap of $6.6 billion.

AI Bubble Signs?: Investor Behavior in Acquisition Landscape

The recent trend of investors rejecting acquisition offers in hopes of more lucrative deals indicates the possibility of an impending AI bubble.

CoreWeave Expands Its Portfolio

In the meantime, CoreWeave continues to pursue acquisitions, recently acquiring Marimo, an open-source competitor to Jupyter Notebook, for an undisclosed amount. PitchBook estimates that Marimo has raised around $5 million.

The Role of Python Notebooks in AI Development

Python notebooks, which combine code, rich media, and explanatory text into single shareable files, are essential for interactive data analysis and AI app development. This acquisition helps CoreWeave advance from simple hosting services to developing AI applications.

Sure! Here are five FAQs addressing the acquisition of Marimo, a Python notebook platform, by CoreWeave’s Core Scientific:

FAQ 1: What is Marimo?

Answer: Marimo is a Python notebook platform designed for data analysis and machine learning. It allows users to write and execute Python code in a collaborative, interactive environment, making it easier to explore datasets and visualize results.

FAQ 2: Why did CoreWeave acquire Marimo?

Answer: CoreWeave acquired Marimo to enhance its data science and machine learning capabilities. By integrating Marimo’s user-friendly notebook environment, CoreWeave aims to provide its clients with more powerful tools for data analysis and innovation, streamlining workflows for data scientists and analysts.

FAQ 3: How will this acquisition impact existing Marimo users?

Answer: Existing Marimo users can expect continued support and enhancements to the platform. CoreWeave plans to integrate additional features and infrastructure improvements, providing a smoother experience, while maintaining the core functionalities that users appreciate.

FAQ 4: What benefits does this acquisition offer to CoreWeave customers?

Answer: CoreWeave customers will gain access to advanced data science tools and a collaborative platform for coding and analysis. The integration of Marimo’s capabilities allows users to leverage cutting-edge machine learning resources directly within their existing workflows, increasing efficiency and innovation.

FAQ 5: Will Marimo remain an independent platform after the acquisition?

Answer: While Marimo will be integrated into CoreWeave’s offerings, it is expected to operate under its own brand. CoreWeave aims to preserve the unique features of Marimo while enhancing its functionalities and user experience in alignment with CoreWeave’s broader business goals.

Source link

The post AI Frenzy Derails CoreWeave’s Acquisition of Core Scientific — Now Acquiring Python Notebook Marimo appeared first on bobweb.ai.

When it comes to working with powerful models and APIs as developers and data scientists, the efficiency and performance of API interactions become essential as applications scale. Asynchronous programming plays a key role in maximizing throughput and reducing latency when dealing with LLM APIs.

This comprehensive guide delves into asynchronous LLM API calls in Python, covering everything from the basics to advanced techniques for handling complex workflows. By the end of this guide, you’ll have a firm grasp on leveraging asynchronous programming to enhance your LLM-powered applications.

Before we dive into the specifics of async LLM API calls, let’s establish a solid foundation in asynchronous programming concepts.

Asynchronous programming allows multiple operations to be executed concurrently without blocking the main thread of execution. The asyncio module in Python facilitates this by providing a framework for writing concurrent code using coroutines, event loops, and futures.

Key Concepts:

• Coroutines: Functions defined with async def that can be paused and resumed.
• Event Loop: The central execution mechanism that manages and runs asynchronous tasks.
• Awaitables: Objects that can be used with the await keyword (coroutines, tasks, futures).

Here’s a simple example illustrating these concepts:

            import asyncio
            async def greet(name):
                await asyncio.sleep(1)  # Simulate an I/O operation
                print(f"Hello, {name}!")
            async def main():
                await asyncio.gather(
                    greet("Alice"),
                    greet("Bob"),
                    greet("Charlie")
                )
            asyncio.run(main())
        

In this example, we define an asynchronous function greet that simulates an I/O operation using asyncio.sleep(). The main function runs multiple greetings concurrently, showcasing the power of asynchronous execution.

The Importance of Asynchronous Programming in LLM API Calls

LLM APIs often require making multiple API calls, either sequentially or in parallel. Traditional synchronous code can lead to performance bottlenecks, especially with high-latency operations like network requests to LLM services.

For instance, consider a scenario where summaries need to be generated for 100 articles using an LLM API. With synchronous processing, each API call would block until a response is received, potentially taking a long time to complete all requests. Asynchronous programming allows for initiating multiple API calls concurrently, significantly reducing the overall execution time.

Setting Up Your Environment

To start working with async LLM API calls, you’ll need to prepare your Python environment with the required libraries. Here’s what you need:

• Python 3.7 or higher (for native asyncio support)
• aiohttp: An asynchronous HTTP client library
• openai: The official OpenAI Python client (if using OpenAI’s GPT models)
• langchain: A framework for building applications with LLMs (optional, but recommended for complex workflows)

You can install these dependencies using pip:

        pip install aiohttp openai langchain
    

Basic Async LLM API Calls with asyncio and aiohttp

Let’s begin by making a simple asynchronous call to an LLM API using aiohttp. While the example uses OpenAI’s GPT-3.5 API, the concepts apply to other LLM APIs.

            import asyncio
            import aiohttp
            from openai import AsyncOpenAI
            async def generate_text(prompt, client):
                response = await client.chat.completions.create(
                    model="gpt-3.5-turbo",
                    messages=[{"role": "user", "content": prompt}]
                )
                return response.choices[0].message.content
            async def main():
                prompts = [
                    "Explain quantum computing in simple terms.",
                    "Write a haiku about artificial intelligence.",
                    "Describe the process of photosynthesis."
                ]
                
                async with AsyncOpenAI() as client:
                    tasks = [generate_text(prompt, client) for prompt in prompts]
                    results = await asyncio.gather(*tasks)
                
                for prompt, result in zip(prompts, results):
                    print(f"Prompt: {prompt}\nResponse: {result}\n")
            asyncio.run(main())
        

This example showcases an asynchronous function generate_text that calls the OpenAI API using the AsyncOpenAI client. The main function executes multiple tasks for different prompts concurrently using asyncio.gather().

This approach enables sending multiple requests to the LLM API simultaneously, significantly reducing the time required to process all prompts.

Advanced Techniques: Batching and Concurrency Control

While the previous example covers the basics of async LLM API calls, real-world applications often demand more advanced strategies. Let’s delve into two critical techniques: batching requests and controlling concurrency.

Batching Requests: When dealing with a large number of prompts, batching them into groups is often more efficient than sending individual requests for each prompt. This reduces the overhead of multiple API calls and can enhance performance.

            import asyncio
            from openai import AsyncOpenAI
            async def process_batch(batch, client):
                responses = await asyncio.gather(*[
                    client.chat.completions.create(
                        model="gpt-3.5-turbo",
                        messages=[{"role": "user", "content": prompt}]
                    ) for prompt in batch
                ])
                return [response.choices[0].message.content for response in responses]
            async def main():
                prompts = [f"Tell me a fact about number {i}" for i in range(100)]
                batch_size = 10
                
                async with AsyncOpenAI() as client:
                    results = []
                    for i in range(0, len(prompts), batch_size):
                        batch = prompts[i:i+batch_size]
                        batch_results = await process_batch(batch, client)
                        results.extend(batch_results)
                
                for prompt, result in zip(prompts, results):
                    print(f"Prompt: {prompt}\nResponse: {result}\n")
            asyncio.run(main())
        

Concurrency Control: While asynchronous programming allows for concurrent execution, controlling the level of concurrency is crucial to prevent overwhelming the API server. This can be achieved using asyncio.Semaphore.

            import asyncio
            from openai import AsyncOpenAI
            async def generate_text(prompt, client, semaphore):
                async with semaphore:
                    response = await client.chat.completions.create(
                        model="gpt-3.5-turbo",
                        messages=[{"role": "user", "content": prompt}]
                    )
                    return response.choices[0].message.content
            async def main():
                prompts = [f"Tell me a fact about number {i}" for i in range(100)]
                max_concurrent_requests = 5
                semaphore = asyncio.Semaphore(max_concurrent_requests)
                
                async with AsyncOpenAI() as client:
                    tasks = [generate_text(prompt, client, semaphore) for prompt in prompts]
                    results = await asyncio.gather(*tasks)
                
                for prompt, result in zip(prompts, results):
                    print(f"Prompt: {prompt}\nResponse: {result}\n")
            asyncio.run(main())
        

In this example, a semaphore is utilized to restrict the number of concurrent requests to 5, ensuring the API server is not overwhelmed.

Error Handling and Retries in Async LLM Calls

Robust error handling and retry mechanisms are crucial when working with external APIs. Let’s enhance the code to handle common errors and implement exponential backoff for retries.

            import asyncio
            import random
            from openai import AsyncOpenAI
            from tenacity import retry, stop_after_attempt, wait_exponential
            class APIError(Exception):
                pass
            @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
            async def generate_text_with_retry(prompt, client):
                try:
                    response = await client.chat.completions.create(
                        model="gpt-3.5-turbo",
                        messages=[{"role": "user", "content": prompt}]
                    )
                    return response.choices[0].message.content
                except Exception as e:
                    print(f"Error occurred: {e}")
                    raise APIError("Failed to generate text")
            async def process_prompt(prompt, client, semaphore):
                async with semaphore:
                    try:
                        result = await generate_text_with_retry(prompt, client)
                        return prompt, result
                    except APIError:
                        return prompt, "Failed to generate response after multiple attempts."
            async def main():
                prompts = [f"Tell me a fact about number {i}" for i in range(20)]
                max_concurrent_requests = 5
                semaphore = asyncio.Semaphore(max_concurrent_requests)
                
                async with AsyncOpenAI() as client:
                    tasks = [process_prompt(prompt, client, semaphore) for prompt in prompts]
                    results = await asyncio.gather(*tasks)
                
                for prompt, result in results:
                    print(f"Prompt: {prompt}\nResponse: {result}\n")
            asyncio.run(main())
        

This enhanced version includes:

• A custom APIError exception for API-related errors.
• A generate_text_with_retry function decorated with @retry from the tenacity library, implementing exponential backoff.
• Error handling in the process_prompt function to catch and report failures.

Optimizing Performance: Streaming Responses

For prolonged content generation, streaming responses can significantly improve application performance. Instead of waiting for the entire response, you can process and display text chunks as they arrive.

            import asyncio
            from openai import AsyncOpenAI
            async def stream_text(prompt, client):
                stream = await client.chat.completions.create(
                    model="gpt-3.5-turbo",
                    messages=[{"role": "user", "content": prompt}],
                    stream=True
                )
                
                full_response = ""
                async for chunk in stream:
                    if chunk.choices[0].delta.content is not None:
                        content = chunk.choices[0].delta.content
                        full_response += content
                        print(content, end='', flush=True)
                
                print("\n")
                return full_response
            async def main():
                prompt = "Write a short story about a time-traveling scientist."
                
                async with AsyncOpenAI() as client:
                    result = await stream_text(prompt, client)
                
                print(f"Full response:\n{result}")
            asyncio.run(main())
        

This example illustrates how to stream the response from the API, printing each chunk as it arrives. This method is particularly beneficial for chat applications or scenarios where real-time feedback to users is necessary.

Building Async Workflows with LangChain

For more complex LLM-powered applications, the LangChain framework offers a high-level abstraction that simplifies the process of chaining multiple LLM calls and integrating other tools. Here’s an example of using LangChain with asynchronous capabilities:

            import asyncio
            from langchain.llms import OpenAI
            from langchain.prompts import PromptTemplate
            from langchain.chains import LLMChain
            from langchain.callbacks.manager import AsyncCallbackManager
            from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
            async def generate_story(topic):
                llm = OpenAI(temperature=0.7, streaming=True, callback_manager=AsyncCallbackManager([StreamingStdOutCallbackHandler()]))
                prompt = PromptTemplate(
                    input_variables=["topic"],
                    template="Write a short story about {topic}."
                )
                chain = LLMChain(llm=llm, prompt=prompt)
                return await chain.arun(topic=topic)
            async def main():
                topics = ["a magical forest", "a futuristic city", "an underwater civilization"]
                tasks = [generate_story(topic) for topic in topics]
                stories = await asyncio.gather(*tasks)
                
                for topic, story in zip(topics, stories):
                    print(f"\nTopic: {topic}\nStory: {story}\n{'='*50}\n")
            asyncio.run(main())
        

Serving Async LLM Applications with FastAPI

To deploy your async LLM application as a web service, FastAPI is an excellent choice due to its support for asynchronous operations. Here’s how you can create a simple API endpoint for text generation:

            from fastapi import FastAPI, BackgroundTasks
            from pydantic import BaseModel
            from openai import AsyncOpenAI
            app = FastAPI()
            client = AsyncOpenAI()
            class GenerationRequest(BaseModel):
                prompt: str
            class GenerationResponse(BaseModel):
                generated_text: str
            @app.post("/generate", response_model=GenerationResponse)
            async def generate_text(request: GenerationRequest, background_tasks: BackgroundTasks):
                response = await client.chat.completions.create(
                    model="gpt-3.5-turbo",
                    messages=[{"role": "user", "content": request.prompt}]
                )
                generated_text = response.choices[0].message.content
                
                # Simulate some post-processing in the background
                background_tasks.add_task(log_generation, request.prompt, generated_text)
                
                return GenerationResponse(generated_text=generated_text)
            async def log_generation(prompt: str, generated_text: str):
                # Simulate logging or additional processing
                await asyncio.sleep(2)
                print(f"Logged: Prompt '{prompt}' generated text of length {len(generated_text)}")
            if __name__ == "__main__":
                import uvicorn
                uvicorn.run(app, host="0.0.0.0", port=8000)
        

This FastAPI application creates an endpoint /generate that accepts a prompt and returns generated text. It also demonstrates using background tasks for additional processing without blocking the response.

Best Practices and Common Pitfalls

When working with async LLM APIs, consider the following best practices:

1. Use connection pooling: Reuse connections for multiple requests to reduce overhead.
2. Implement proper error handling

   1. What is an Asynchronous LLM API call in Python?
      An asynchronous LLM API call in Python allows you to make multiple API calls simultaneously without blocking the main thread, increasing efficiency and speed of your program.

   2. How do I make an asynchronous LLM API call in Python?
      To make an asynchronous LLM API call in Python, you can use libraries such as aiohttp and asyncio to create asynchronous functions that can make multiple API calls concurrently.

   3. What are the advantages of using asynchronous LLM API calls in Python?
      Using asynchronous LLM API calls in Python can significantly improve the performance of your program by allowing multiple API calls to be made concurrently, reducing the overall execution time.

   4. Can I handle errors when making asynchronous LLM API calls in Python?
      Yes, you can handle errors when making asynchronous LLM API calls in Python by using try-except blocks within your asynchronous functions to catch and handle any exceptions that may occur during the API call.

   5. Are there any limitations to using asynchronous LLM API calls in Python?
      While asynchronous LLM API calls can greatly improve the performance of your program, it may be more complex to implement and require a good understanding of asynchronous programming concepts in Python. Additionally, some APIs may not support asynchronous requests, so it’s important to check the API documentation before implementing asynchronous calls.

   Source link

   The post A Comprehensive Guide to Making Asynchronous LLM API Calls in Python appeared first on bobweb.ai.

   ]]> https://bobweb.ai/a-comprehensive-guide-to-making-asynchronous-llm-api-calls-in-python/feed/ 0